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Introduction 


The Common Core State Standards for Mathematics (CCSSM) were developed as part of a 
broader effort coordinated by the National Governors Association Center for Best Practices and 
the Council of Chief State School Officers to develop common state standards in key subjects. 
Released in 2010, the CCSSM are intended to provide a blueprint for the mathematics that 
students should learn, and when they should learn that mathematics, to ensure that they are well 
prepared for college and careers when they graduate from high school. 

States, districts, schools, and teachers may choose to follow, or not follow, that blueprint; and 
those that do will likely differ in how they interpret it, what they plan to do about it, and their 
capacity to implement their plans. As a result, and because of considerable differences among 
school contexts and student populations, common standards are not likely to lead to the same 
results, either in implementation or in impacts on students. Clearly, it would not be feasible to 
study in depth how the CCSSM are being interpreted and how they are influencing every part of 
the mathematics education system, nor to examine the effects of that influence on mathematics 
teaching and learning in every classroom in the United States. Our goal in this project was to 
suggest a set of priorities for research that would provide the field with a reasonably broad and 
deep understanding of the influence of the CCSSM, including whether, how, and under what 
conditions the initiative results in improved student learning in grades K-12 and enhanced 
college/career readiness. 

This report provides background information on the CCSSM and on a framework for 
investigating the influence of standards developed by a committee of the National Research 
Council. The report then describes how we went about developing a priority research agenda; 
and concludes with the resulting recommendations for research on the influence of the CCSSM. 


Background on the CCSSM 

To help ensure that all students will be prepared to succeed in society and in our global economy, 
the CCSSM were intended to align with expectations for both college and career readiness. 

These expectations include studying rigorous content benchmarked to international standards; 
focusing on deeper study of fewer topics than has generally been the case in mathematics 
education in the US; and attending to coherence by connecting ideas within and across topics. 
Further, the expectations include application of mathematical knowledge through higher order 
skills. The CCSSM were developed by a lead author team with expertise in mathematics 
content, teaching, and education research. Their charge was to write standards based on three 
kinds of evidence: evidence from high performing states and countries; findings from cognitive 
science and mathematics education research, including student achievement studies; and lessons 
learned from standards-based accountability systems. The goal was to develop a set of standards 
that attends to the demands of college and career readiness; is reasonable in scope, 
developmentally appropriate, and instructionally manageable; and provides sufficient guidance 
for the design of curricula and instructional materials. 
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As part of the writing process, the author team received feedback on successive drafts of the 
standards from various stakeholders including mathematicians, mathematics educators, school 
administrators, community and parent organizations, members of the business community, civil 
rights groups, and representatives from the state education agencies, and governors’ offices. 

The initiative appointed a validation committee of 25 members who, as a group, had content 
expertise, and experience in developing and implementing mathematics education standards. 

This committee reviewed the development process, including the use of evidence from prior 
research, and provided feedback at various points throughout the process. 

The CCSSM are intended to address both conceptual understanding of mathematics and 
procedural skills. For grades K-8, the content standards are written for individual grades, rather 
than in grade bands, as had been the case in the NCTM curriculum standards. (National Council 
of Teachers of Mathematics, 1989, 2000) The high school standards are organized by 
“conceptual category” rather than by course, with two proposed course organizations included in 
an appendix to the standards document. The CCSSM also include eight standards for 
mathematical practice to be addressed throughout K-12 mathematics education, describing 
fundamental approaches to, and dispositions toward, learning and doing mathematics. 

(Examples include constructing viable arguments and critiquing the reasoning of others, 
attending to precision, and looking for and making use of structure.) 

In addition to the importance of focusing on a smaller number of more rigorous content 
standards than the standards most states had previously adopted, the developers emphasize the 
need for coherence, including making “optimal use of what was learned before [and teaching} 
today’s mathematics in an optimal way to support extensions to tomorrow’s mathematics.” The 
idea of “standards progressions” is central to the CCSSM, describing the content to be addressed 
as well as the connections to be made both within and across grade levels. As one of the author 
team has noted, not only are the individual content standards “carefully crafted and worth 
individual attention” the structure of the CCSSM as a set is important; “the Standards as a whole 
are also a vision for a coherent mathematical education, and as such the document is more than 
the sum of its parts.” (Zimba, 2011, p 4.) 

Although states are expected to adopt the CCSSM in their entirety, adopting states have the 
option to include up to 15 percent additional standards at their discretion. Because there is no 
agreed-upon way to quantify the standards in the CCSSM, it is unclear how one would determine 
what constitutes an additional 15 percent, but in spirit it seems that most mathematics instruction 
should be based on the CCSSM, with a small portion potentially addressing additional content 
deemed important at the state level. 

Even before states began to make decisions about the adoption of the Common Core 
State Standards, a diverse and lengthy list of groups had expressed support for the 
initiative, including, for example, the Association for Governing Boards of Universities 
and Colleges, the Council of Great City Schools, GlaxoSmithKline, and the National 
PTA (Common Core State Standards Initiative, 2010a). 
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Within a year of the June 2010 release of the CCSSM, more than 40 states and the District of 
Columbia had adopted the CCSSM as their state standards; textbook publishers were exploring 
issues of alignment; professional associations were providing opportunities for their members to 
learn about the standards; and eight key organizations formed the Mathematics Common Core 
Coalition (MC ) to help ensure successful communication, interpretation, implementation, and 
assessment of these standards. Additionally, the Department of Education provided substantial 
funding to two consortia of states to develop comprehensive assessment systems aligned with the 
CCSSM. It is likely that the CCSSM will be an important factor in mathematics education 
nationally for the foreseeable future. 

Understanding what actually happens in response to the CCSSM, and why, can help inform both 
efforts to implement these standards and initiatives to develop and implement future sets of 
standards. The purpose of our project was not just to consider the research that might be 
conducted as part of understanding the influence of the CCSSM. Rather, our goal was to identify 
a priority research agenda for what should be studied to learn as much as possible given the 
inevitable limitations of time and resources for research. 


A Framework for a Research Agenda 

The process used to develop the CCSSM included looking at the standards of high-performing 
countries, and considering what students would need to know and be able to do in order to be 
ready for post- secondary pursuits. However, the desired focus and coherence would not be 
possible if every suggested learning goal was included; decisions needed to be made about what 
to include as core. Consequently, the content of the CCSSM depended on value judgments about 
what knowledge is most important. In contrast, assessments of the impact of a given set of 
standards on the education system can be based firmly on empirical evidence. The goal of our 
project was to suggest a research agenda to produce the kinds of evidence needed to determine 
and understand the effects of the CCSSM on mathematics education in the United States. 

In considering research for monitoring the effects of the CCSSM, we had the benefit of being 
able to build on earlier work focused on research on national standards. A committee convened 
by the National Research Council (NRC) developed a framework for research on the influence of 
national standards documents (Weiss, Knapp, Hollweg, & Burrill, 2002). The NRC framework 
is based on the recognition that standards documents are unlikely to have a direct impact on 
student learning, but come to influence teaching and learning by first influencing key 
components of the education system. (See Figure 1.) 
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National Research Council Framework for Research 
on the Influence of National Standards Documents 


How has the system responded to the 
introduction of nationally developed 
standards? 


What are the 
consequences for 
student learning? 


Contextual 

Forces 

• Politicians and 

Policy Makers 

• Public 

• Business and 

Industry 

• Professional 

Organizations 


Channels of Influence 
Within the Education System 

Curriculum 

• State, district policy decisions 

• Instructional materials development 

• Text, materials selection 

Teacher Development 

• Initial preparation 

• Certification 

• Professional development 

Assessment and Accountability 

• Accountability systems 

• Classroom assessment 

• State, district assessment 

• College entrance, placement practices 


Teachers 
and Teaching 
Practice in 
classroom 
and school 
contexts 


Within the education system and in its context— 

• How are nationally developed standards being received 
and interpreted? 

• What actions have been taken in response? 

• What has changed as a result? 

• What components of the system have been affected and how? 


Among teachers who 
have been exposed to 
nationally developed 
standards — 

• How have they received 
and interpreted those 
standards? 

• What actions have they 
taken in response? 

• What, if anything, 
about their classroom 
practice has changed? 

• Who has been 
affected and how? 


Student 

Learning 


Among students 
who have been 
exposed to 
standards-based 
practice — 

• How have student 
learning and 
achievement 
changed? 

• Who has been 
affected and how? 


Figure 1 


Classroom instruction across the United States is significantly affected by decisions made at the 
federal, state, district, school, and teacher levels, as well as actions taken by textbook publishers 
and others who are not part of the formal governance structure for public education. The NRC 
committee identified three “channels” through which reforms such as standards might exert their 
influence on education policies, programs, and practices as they make their way (or fail to make 
their way) into K-12 mathematics classrooms: the curriculum channel, the teacher development 
channel, and the assessment and accountability channel. 

For example, a set of standards might impact the curriculum by influencing the nature of 
state/district standards, the content of textbooks and other instructional materials, and the criteria 
for state and local instructional materials adoption. Alternatively, or in addition, a set of 
standards might affect how teachers are prepared and supported by influencing state certification 
policy, teacher preparation and professional development programs, and teacher evaluation 
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systems. Similarly, a set of standards might impact how student learning is assessed, both 
formally and informally, and the design of state and local accountability systems. And, of 
course, these channels interact with one another, for example, what is assessed, particularly when 
there are high stakes involved, in turn impacts how teachers are prepared, supported, and 
evaluated; what is included in textbooks; decisions teachers make about what is taught and how; 
etc. 

The NRC report notes that, eventually, any set of education standards will be judged effective “if 
resources, requirements, and practices throughout the system align with the standards and if 
students in standards-based classrooms demonstrate high achievement in knowledge and skills 
deemed important.” (Weiss et al., 2002, p. 83) Of course, given the variability in the education 
system in the United States, we can anticipate that the CCSSM will exert their influence to a 
different extent and in different ways in particular states (and districts, and schools, and 
classrooms), and for different students, depending on capacity, resources, stakeholder beliefs, 
and a host of other factors. Accordingly, judgments about the effectiveness of a set of standards 
need to consider if there is differential impact in various contexts and on various subpopulations. 

The framework for research developed by the NRC committee highlights several questions that 
need to be answered in order to understand the influence of a set of standards, questions that can 
be applied to any of a number of key components of the education system: 

1. How are the standards being received and interpreted? 

2. What actions have been taken in response to the standards? 

3. What has changed as a result? 

4. Who has been affected and how? 

Although the NRC framework is intended to be applicable to a wide range of standards, it seems 
clear that priorities for what should be investigated for a particular set of standards depend both 
on the nature of those standards and on the theory of action underlying their development and 
implementation. In the case of the CCSSM, both content standards and standards for 
mathematical practice are specified, with emphasis on the need to have students make use of the 
mathematical practices, e.g., constructing viable arguments, as part of learning the designated 
mathematics content. Consequently, judging the influence of the CCSSM would likely need to 
include understanding the extent to which the notion of incorporating the mathematical practices 
is permeating the system, e.g., determining the nature and extent to which teacher preparation 
and professional development programs give teachers an opportunity to leam mathematics 
content through these practices; curriculum developers provide opportunities for students to do 
the same; assessments test students’ abilities with respect to the mathematical practices as well as 
content; and teachers focus on both the content and mathematical practices standards in their 
classrooms. Similarly, judging the influence of the CCSSM would likely need to include 
understanding the extent to which the system becomes more focused (addressing the content 
included in the standards and dropping the content that is no longer called for at a particular 
grade), and more coherent (addressing the notion of mathematics as a progression of ideas.) 

In the past, even states that had roughly equivalent standards by grade band might have 
addressed those standards in different grades, so states typically worked fairly independently to 
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develop instructional materials adoption protocols, professional development programs, etc. The 
promulgators of the Common Core State Standards envisioned that having the same grade-by- 
grade standards would encourage collaboration among states, which in turn would enable 
existing capacity to be used to greater effect. Consequently, as part of understanding how the 
CCSSM permeate the system, research should consider the nature and prevalence of these 
collaborations, and whether they are in fact leading to higher quality products in a more cost- 
effective fashion than was the case when individual states worked independently. At the same 
time, research should consider whether the economies of scale notion is resulting in so little 
variation in approach that it interferes with learning about what works, for whom, under what 
conditions. 

And as a final example, the federal government has provided support for the initiative in the 
form of incentives for states to adopt the standards as part of the Race to the Top competition, 
and funding of two assessment consortia. Investigating the interplay between the state and 
federal “forces” is an appropriate part of understanding the influence of the CCSSM on 
mathematics education in the United States. 


Organization of this Report 

The following section describes how our project went about the development of a priority 
research agenda for understanding the influence of the CCSSM. Subsequent sections describe 
the review process for the draft report and summarize the feedback we received. The final 
section of the report provides the priority research agenda that resulted, and a discussion of some 
methodological considerations for research to understand the influence of the CCSSM. 


Project Activities 

As a precursor to developing a priority research agenda, we needed to have an explicit 
understanding of how the CCSSM might come to make a difference in the system, identifying 
both what the promulgators hoped would happen, and what they and critics of the initiative were 
concerned might happen. We also needed to consider indicators of influence-what would count 
as evidence that the anticipated changes had in fact occurred. And finally, we needed to consider 
which of the many components of the system that could be studied were most important to study, 
and when, in order to understand the influence of the CCSSM. 

As noted in Table 1, multiple data collection strategies were used to collect and synthesize this 
information and to draft research priorities, including: 

1. Review of Common Core State Standards documents (Common Core State Standards 
Initiative, 2010a, 2010b, 2010c, 2010d), and interviews with the authors of the 
CCSSM to understand the intent of these standards; 
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2. Interviews with representatives of the groups that are coordinating the Common Core 
State Standards Initiative to understand how they intended the CCSSM to influence 
the system; 

3. Meetings of small groups of mathematics education and policy researchers to 
consider a conceptual framework for research on the influence of the CCSSM; 

4. Surveys of mathematics educators to get input on how the CCSSM are likely to 
influence the system, both positively and negatively; and 

5. A two-day conference of mathematics education and policy researchers to consider 
priorities for research. 

Throughout the process, our project had the benefit of guidance from an advisory board of 12 
experts in mathematics education and policy research. Their help included suggesting people to 
be interviewed, reviewing interview protocols, and piloting data collection strategies. 


of t 


Table 1 

A Priority Research Agenda for Understanding the Influence 
he Common Core State Standards for Mathematics Methodology 


Process for 
Developing a 
Priority 
Research 
Agenda 

Data Collection Method 

Review 

Standards 

Documents 

Interviews 

Surveys of 
Mathematics 
Educators 

Convening Experts 

Authors 

of 

CCSSM 

Coordinators 

ofCCSS 

Initiative 

Small group 
meetings of 
mathematics 
education 
and policy 
researchers 

Conference 

of 

mathematics 
education 
and policy 
researchers 

Understanding 
of how the 
CCSSM might 
come to make a 
difference in the 
system 

X 

X 

X 

X 

X 

X 

Identifying 

potential 

unintended 

consequences 


X 

X 

X 

X 

X 

Developing 
indicators of 
alignment with 
the CCSSM 


X 

X 

X 

X 

X 

Considering 
priorities for 
research 


X 

X 

X 

X 

X 

Identifying 
methodological 
considerations 
for research on 
understanding 
the influence of 
the CCSSM 





X 

X 
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Review of Common Core State Standards Documents 


In order to glean information about the intent of the CCSSM, project staff reviewed documents 
available on the Common Core State Standards Initiative website (www.corestandards.org): the 
Common Core State Standards in Mathematics, the common core endorsing partners, frequently- 
asked-questions, and standards- setting considerations (Common Core State Standards Initiative, 
2010a, 2010b, 2010c, 2010d). In addition, project staff reviewed the Curriculum and Evaluation 
Standards for School Mathematics and the Principles and Standards for School Mathematics 
(National Council of Teachers of Mathematics 1989, 2000) in relation to the CCSSM in order to 
examine the key differences between the two documents. 


Interviews with Authors and Sponsors of the CCSSM 

We anticipated that the promulgators of the CCSSM might well have additional ideas about the 
process that were not made explicit in publicly-available documents. Accordingly, an initial 
project activity was a series of interviews with four key individuals in the National Governors 
Association, the Council of Chief State School Officers, and Achieve, as well as with the three 
members of the core writing team. The purpose of the interviews was to collect additional 
information about how the CCSSM might make a difference the system and the potential 
intended and unintended consequences that might result. Interviewees were asked about why the 
initiative focused on the state level, and how they anticipated the standards would influence 
various aspects of the system, e.g., curriculum materials, colleges/universities, schools/districts, 
and federal policies. They were also asked what they thought would need to happen to ensure 
that the standards were implemented meaningfully and how we would know if the CCSSM were 
being implemented as intended. Further, those interviewed were asked what they would look for 
in two, five, and ten years that would indicate the CCSSM were having the intended impact. 


Surveys of Mathematics Educators 

In addition to getting the perspective of the promulgators of the CCSSM, we wanted to be sure to 
get the views of the broader mathematics education community about what might happen as a 
result of the release of the CCSSM. Accordingly, we asked the project advisory board to help us 
identify a diverse group of mathematics educators to be queried about how these standards might 
influence various components of the system, including both people who have indicated support 
for the CCSSM and those who have expressed concerns about the initiative. 

A total of 32 mathematics educators completed electronic surveys in February 2011, including 
university faculty; state and local district mathematics supervisors; and people employed in 
research and development organizations. Each individual responded to a series of questions 
about the potential influence and possible unintended consequences of the CCSSM in an 
assigned area, i.e., curriculum, teacher development, or assessment/accountability. 
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Mathematics educators were queried about how alignment might be operationalized in their 
assigned area. First, respondents were asked to react to descriptions that we drafted of what a 
key component of the mathematics education system might look like if closely aligned with the 
CCSSM. They indicated whether they agreed with our description of alignment, and modified 
the description if needed. Second, respondents were asked to provide their own descriptions of 
what classroom instruction and various other components of the system would look like when 
closely aligned with the CCSSM. For example, the curriculum channel group reacted to 
descriptions of CCSSM-aligned curriculum materials for students and associated guidance for 
teachers. They also provided descriptions of what alignment to the CCSSM would look like in 
the classroom, in curriculum materials review criteria, in curriculum-related activities of 
professional mathematics and mathematics education societies, and in federal regulations and 
solicitations regarding curriculum. 

Finally, mathematics educators were queried about what research in their particular channel of 
expertise they would prioritize in the short- and long-term in order to understand the influence of 
the CCSSM. 


Convening Experts: Small Group Meetings of Mathematics Education and 
Policy Researchers 

Mathematics education and policy researchers had significant input into the various aspects of 
developing a priority research agenda. Early in the process, two meetings with small groups of 
mathematics education and policy researchers were conducted to consider how the NRC 
framework for investigating the influence of standards might need to be modified in developing a 
conceptual framework specific to the CCSSM. In addition, we solicited input from these groups 
on what types of research would be most important to conduct at different levels of the system 
and different time points. 


Convening Experts: Conference of Mathematics Education and Policy 
Researchers 

A culminating event in the project was a two-day conference of 30 mathematics education and 
policy researchers to consider priorities for research, recognizing a need to be strategic in the 
deployment of research resources. For example, although standards might affect any part of the 
mathematics education system, there are some components of the system that will be particularly 
important in determining the extent to which the CCSSM are likely to go to scale, so early 
research should focus on influence on those components. Similarly, research needs to take into 
account the fact that the CCSSM represent fairly minor changes from current state standards for 
some mathematics topics, and substantial changes in approach and/or grade placement for others. 
The more major changes may be good targets for broad studies. But seemingly subtle changes 
may in fact, have important mathematical implications that are vital to student learning along a 
standards progression. These more nuanced differences may be glossed over if the changes they 
represent are not recognized and highlighted in particular parts of the system, such as curriculum 
materials, professional development, and assessment. 
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Initial data collection activities had identified pre-service education, both traditional and 
alternative preparation programs, as a key leverage point for the CCSSM; the idea was that well- 
designed preparation programs would avoid the need for the kinds of “remediation” that will be 
necessary for current teachers who have not had the opportunity to learn the mathematics 
content/mathematical practices called for in the CCSSM. The fact that so many states have 
adopted the same standards means that guidance documents, for example the revision of The 
Mathematical Education of Teachers (Conference Board of the Mathematical Sciences, 2001) 
will be able to provide more detailed, broadly-applicable guidance than has been possible before. 
And modifications made to a program in one state may be able to be used in multiple states, 
increasing the likelihood of changes at scale within the available resources. 

The preparation of new teachers was used at the research agenda conference to illustrate the 
process of developing research foci for understanding the influence of the CCSSM. We 
provided a concept map illustrating potential pathways of influence of the CCSSM in the 
preparation of new teachers. (See Appendix A.) We then asked small groups to consider what 
“pieces” within this map would be most important to study and why. This process enabled the 
participants to develop a shared vision of the “products” to be drafted during the conference, 
which would include not only suggested priority research questions, but also the rationale for 
their recommendations. 

Prior to the conference, each participant had selected one of the “channels of influence” 
identified in the NRC framework — curriculum, teacher development, or assessment/ 
accountability — and grade range preference for initial focus. Participants were assigned to 
working groups according to their selections, given a concept map of the designated channel, and 
asked to revise the map to fit the group’s conceptualization of how the CCSSM might exert their 
influence. Groups then discussed the maps and selected targets they considered high priorities for 
research, describing both their rationale for those selections and the nature of the recommended 
research. After the groups discussed their choices and rationales, the elementary and secondary 
groups within channels met and compared their work; in some cases they choose to modify their 
recommendations to be more consistent across grades, and in other cases they decided that 
different research foci by grade range were appropriate. 

At the end of the first day, we called attention to several components of the system/key issues 
that we believed were in particular need of additional consideration: classroom practice, student 
outcomes, access and equity, stakeholder support, state transition strategies, and learning 
progressions. Participants suggested an additional area as well: the capacity of the mathematics 
teaching community. Individuals could sign up to continue to consider research on the influence 
of the CCSSM in one of the three channels as outlined in the concept maps, or to focus on one of 
these additional areas. Alternatively, they could choose to explore the development of a data 
infrastructure, an area of need that was suggested by a participant as extremely important for 
research, not only on the influence of the CCSSM but in monitoring changes and managing 
research priorities in the system more broadly. 

After considering research priorities in selected areas of their choice, participants had the 
opportunity to hear the results of all of the groups’ deliberations. As the final conference 
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activity, participants were asked to reflect on the discussions over the course of the meeting and 
describe how they would apportion research resources among the proposed areas, both in the 
very near term and after 2014, when the consortia assessments are due to be released. 


Considerations in the Development of 
a Priority Research Agenda 

The sections below provide information gathered through these data collection activities in 
relation to: (1) how the CCSSM are intended to influence the mathematics education system; (2) 
possible unintended consequences; (3) indicators of alignment with the CCSSM; and (4) priority 
lines of inquiry. 


How the Common Core State Standards for Mathematics Are Intended to 
Influence the System 

The CCSSM represent different kinds of shifts in some content areas, but are quite similar to 
prevailing practice in others. 1 As a consequence, research on some content areas will be more 
telling than others regarding the influence of the CCSSM. For example, mathematics educators 
have noted that the CCSSM provide greater emphasis on the meaning of operations with whole 
numbers and rational numbers throughout the elementary grades than did many previous sets of 
standards; consequently, investigations of alignment might look for increased attention to these 
topics in curriculum materials, teacher preparation and professional development, and classroom 
assessment. Conversely, the CCSSM provide less emphasis on probability and early algebraic 
patterning than earlier sets of standards, especially in particular states, so alignment would result 
in comparable reductions in emphasis on these topics throughout relevant parts of the system. 
Still other topics, such as similarity and congruence of geometric figures, are treated quite 
differently in the CCSSM compared to most state mathematics standards that were in place at the 
time of their release. 

Topics placed at the same grade levels as they have been in most previous sets of standards may 
be particularly important to study if there are subtle differences in approach that in actuality have 
crucial mathematical implications within the CCSSM standards progressions. For example, in 
the third grade, students’ understanding of fractions is intended to directly extend from their 
understanding of multiplication of whole numbers, such that fractions in the form a/b (both 
proper and improper), are to be understood as iterations of a unit fraction ( a iterations of 1/b ), 
leading to a later understanding of a/b as a product of a whole number a and a unit fraction 1/b. 
Alignment for such topics would involve a shift that reflects the foundations and approaches of 
the CCSSM. 


1 We are indebted to Chris Hirsch, Glenda Lappan, and Barbara Reys for conducting comparisons of the CCSSM to 
recent state standards. The specific examples used throughout this document to illustrate differences between 
CCSSM and other standards are drawn from their work, and from feedback provided by reviewers of the draft 
report. 
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Taken as a whole, the CCSSM provide a blueprint for the mathematics that students should 
learn, and when they should learn that mathematics, with the expectation that following this plan 
will enable K-12 students to gain the mathematical knowledge that will ensure that they are 
college and career ready at the time of graduation from high school. A research agenda on 
understanding the influence of the CCSSM should take into consideration not only the expected 
outcome, but also how the standards are intended to influence the system in order to produce 
those outcomes. 

Available documents describe a number of elements of how the CCSSM are expected to operate 
to add clarity, coherence, and focus to the mathematics education system, helping to ensure that 
students in every school, in every state, are receiving a high quality education. For example: 

The standards clearly communicate what is expected of students at each grade level. 

This will allow our teachers to be better equipped to know exactly what they need to help 
students learn and establish individualized benchmarks for them. The Common Core 
State Standards focus on core conceptual understandings and procedures starting in the 
early grades, thus enabling teachers to take the time needed to teach core concepts and 
procedures well — and to give students the opportunity to master them. . . . With students, 
parents and teachers cdl on the same page and working together for shared goals, we can 
ensure that students make progress each year and graduate from school prepared to 
succeed in college and in a modern workforce. (Common Core State Standards 
Initiative, 2010c) 

Having common standards across states is also expected to facilitate collaboration throughout the 
system. For example: 

The Common Core State Standards will enable participating states to work together to: 
o Make expectations for students clear to parents, teachers, and the general public; 
o Encourage the development of textbooks, digital media, and other teaching 
materials aligned to the standards; 

o Develop and implement comprehensive assessment systems to measure student 
performance against the Common Core State Standards that will replace the 
existing testing systems that too often are inconsistent, burdensome and 
confusing; and 

o Evaluate policy changes needed to help students and educators meet the 
standards. (Common Core State Standards Initiative, 2010b) 

In particular, standards documents suggest that having common standards will “provide a greater 
opportunity to share best practices that will improve the ability of the system to best serve the 
needs of all students, including young people with disabilities and English language learners” 
(Common Core State Standards Initiative, 2010b). 

The emphasis in the standards documents is on communicating a vision for a coherent approach 
to teaching the mathematics that has been identified as important, not on describing the processes 
by which this vision would come to fruition. Interviews with authors and coordinators of the 
CCSSM provided additional information on standards and their possible influence. A number of 
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themes emerged from these interviews. First, there was an emphasis on the importance of the 
states as the key decision-makers in this initiative, as the state is the entity that is 
“constitutionally charged, not the federal government, to create an adequate educational system, 
particularly in K-12.” Interviewees noted that the fact that the common standards were “state- 
driven” was essential in getting state buy-in. 

Second, although they highlighted different components of the work as particularly important, 
the promulgators consistently stressed that state adoption was only the beginning, “simply a 
verbal commitment to pursue a path of implementation.” For example, interviewees noted that 
in order to successfully implement the CCSSM, states will have to evaluate their policy 
landscapes to ensure that they are providing appropriate, coherent guidance and incentives. They 
also stressed the need for states to prepare both current and prospective teachers to teach the 
CCSSM, and to ensure that they have access to high quality curriculum materials aligned with 
the standards. 

Third, the interviewees described the advantages they expected from common standards in 
addressing the various challenges, including the opportunity to collaborate in developing 
programs and tools that can be broadly used, so “you don’t need as many efforts.” They noted 
that in the future common standards will mean that students will have less difficulty when they 
move from one state to another, and suggested that the combination of common standards and 
common assessments will make it easier to conduct research to identify effective programs and 
practices. 


Possible Unintended Consequences 

A priority research agenda needs to consider not only how the CCSSM are intended to influence 
mathematics education, but possible unintended consequences as well. Perspectives on what 
might happen as a result of the release of the CCSSM were collected in surveys administered to 
an external group of mathematics educators, as well as the interviews of people involved with the 
development and coordination of the CCSSM. Some of the mathematics educators we queried 
do not expect the system to become aligned around this set of mathematics standards, whether 
due to lack of capacity, uneven commitment, and/or inadequate resources. Others expressed 
concern about the consequences if the system does in fact align around the Common Core State 
Standards, either because of inadequacies they perceive in this particular set of standards and/or 
problems they see with the idea of common standards, or any set of standards. 

It is important to note that concerns about what might happen in response to the CCSSM were 
expressed both by “insiders” in the Common Core State Standards Initiative, and people who 
were not involved in their development. The promulgators were clearly surprised that so many 
states had adopted the Common Core State Standards so quickly, attributing it in part to the fact 
that the federal government had provided incentives for adoption as part of the Race to the Top 
competition. Interestingly, the interviewees had somewhat different perspectives on the rapid 
pace of state adoption. For example, one person cited the fact that more than 40 states had 
adopted the CCSSM within months of their release as a major positive sign, noting that this 
reception was “beyond our wildest dreams.” In contrast, several of the people who were 
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instrumental in the Common Core State Standards Initiative expressed concern that the initiative 
would lose the opportunity to leam from a small number of pilot states before moving to broad 
scale implementation, e.g.: “I hope that the quickness on the front end doesn’t result in the lack 
of information on the back end.” 

Members of the writing team also cautioned about the consequences of implementation 
proceeding without adequate consideration to the intent of the CCSSM. Said one, “if . . . you 
read the standards by not reading them but by reducing them to a list of topics by grade level, 
you will miss the point. What a standard says about a topic matters a lot.” 

Several mathematics educators we queried who were external to the effort expressed concern 
about what the standards do not say, indicating that there is not enough emphasis in particular 
areas they deemed important. For example, even though “use appropriate tools strategically” is 
included as a practice standard, several respondents indicated there was not sufficient attention to 
technology in the CCSSM, e.g., the CCSSM “do not mention calculators once in grades K-8.” 
The consequence, some predict, will be that students will not be sufficiently prepared to use 
mathematics tools in the “real world.” A number of respondents whose work is focused at the 
secondary level questioned whether the CCSSM’ s emphasis on certain content topics, and de- 
emphasis on others, might favor college-readiness, especially in preparation for careers in STEM 
fields, over career-readiness and informed citizenship. For some, the issue of appropriate content 
priorities, especially for high school, was exacerbated by the July 2011 release of a draft 
curriculum framework (Partnership for Assessment of Readiness for College and Careers, 2011) 
for assessment that placed statistics and probability in the low priority category, and algebra, 
procedural fluency, and functions in the top category. 

A number of the mathematics educators we queried noted that the CCSSM had been, in 
the words of one “parachuted in,” suggesting that the initiative would have benefitted 
considerably from more input from the broader mathematics education community. 

There is particular concern about the standards progressions, with several respondents 
noting that the expectations are not sufficiently grounded in what research on learning 
has revealed about students’ thinking and mathematical capabilities, and that standards 
progressions that seem sensible from the perspective of the discipline may not be sensible 
from a learning perspective. For example: 

What the CCSS-M provides is a progression of content with little attention to what 25 
years of research has shown about learning. 

1 am somewhat dubious about curricular progressions . . . once you get past very 
elementary mathematics, such as the arithmetic of whole numbers. (There are many 
productive pathways to proficiency with fractions, for example, or other complex 
domains .) 

Still others noted that the lack of guidance for instruction in the CCSSM could prove 
problematic. For example, there was concern that teachers might think that because a standard 
uses a formula to describe a mathematics idea, teachers might mistakenly infer that the formula 
should be explicitly taught in that way rather than providing opportunities for students to develop 
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the concept. Noted one respondent: “I think these standards are particularly susceptible to that 
[kind of interpretation.]” 

Similarly, some of the mathematics educators we queried suggested that while one can certainly 
argue that some form of the standards for mathematical practice are appropriate at every grade, 
the lack of nuance about what they might look like at different grades may lead to inappropriate 
implementation. Said one: 

There may be some progressions with regard to some of the practices (one can see going 
from informal argumentation to mathematical proof for example), but a lot more is 
simply the question of what’s developmentally appropriate (e.g., you might ask high 
school students to do 15 minutes of independent work, but not 3 rd graders.) 

Other respondents suggested that if states do in fact add 15 percent more material to the CCSSM, 
any benefits associated with having common standards will be lessened. Said one: 

This issue of deciding what content is the core 3 to 5 topics that “must be” at a given 
grade level is key to the development of an evaluable curriculum and its implementation. 
It is also key to addressing the student mobility issue that the U.S. faces, perhaps greater 
than any other developed nation. Thus, the addition of potentially other clutter to the 
curriculum to dilute the focus on the core curriculum gocds will add “noise” to the 
situation and ’’hide ” the valuable information and outcomes desired. 

A general theme among both the promoters of the CCSSM and the mathematics educators we 
queried who are external to the process was that if the CCSSM initiative resulted in only 
superficial, cosmetic changes, we will not see improved student performance. For example, 
when asked about what would count as evidence that the CCSSM were having the desired effect, 
having instructional materials that are more than superficially aligned with the standards was 
frequently cited. Said one key player: “I would be looking for changes in textbooks that go 
beyond some textbook publisher slapping a label on it that says ‘aligned with the Common Core’ 
but not reducing the number of pages or adding coherence.” Similarly, the developers expressed 
concern that when teachers (and textbook publishers) see a topic that was previously included in 
their state standards at the same grade level, they may ignore the fact that the CCSSM calls for a 
different, in some cases radically different, way of approaching that content. 

Quite a few respondents described concerns related to the consortia assessments. Both the 
promulgators of the CCSSM and the people we queried who are external to the development 
process are worried about what will happen if the consortia assessments fall far short of 
expectations. For example, concern was expressed about instruction focusing on minutiae, and 
mathematical literacy being sacrificed, if the assessments fail to incorporate the Standards for 
Mathematical Practice, and instead focus mostly on “skills;” try to “cover” everything; measure a 
myriad of “special techniques” but not student ability to use general purpose tools. 

A number of the mathematics educators we queried who were not involved in the development 
of the CCSSM cited a possible unintended consequence of the consortia assessments at the 
secondary level. Project documents describe two possible pathways for secondary-level 
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mathematics programs, a traditional sequence of Algebra 1, Geometry, and Algebra 2, and an 
integrated sequence of three courses that include algebra, geometry, and statistics in each. 

Neither option is intended to be favored over the other, but the content make-up and timing of 
assessments could result in one approach being more practicable than the other. 

Several respondents expressed concern that the CCSSM will have a dampening effect on 
innovation in content, curriculum materials, and assessment, and will consequently limit the 
ability of the field to generate knowledge in order to make further improvements in mathematics 
education. For example: 

Devoting all our national curriculum development resources to “aligning” with the 
CCSS-M makes little sense. Some national energy should go into the development of 
curriculum [materials] that explicitly do not follow these standards but have other claims 
to validity. We should see what happens when good developers fundamentally rethink 
content pathways in novel ways. This is particularly reasonable since comparative 
effects on learning should be possible via the common assessments that are coming. What 
I am ccdlingfor is some planned curricular experimentation. 

And one respondent extended that concern to teachers, noting that common 
standards/assessments across states will increase the pressure to teach to the tests, and 
discourage teachers from spending time on mathematics that they have found particularly 
motivating to their students. 

For instance, some middle school teachers like to have students create original 
tessellations; some high school teachers like to have students investigate new theorems in 
geometry. ... [The result may be] less teaching of the applicability, power, and beauty of 
mathematics [and] more of a dislike for the subject (as is the case in some of the Pacific 
Rim countries .) 

Finally, there is a great deal of concern about equity. The CCSSM were written to provide a 
definition of readiness for college and career success in relation to K-12 mathematics learning as 
a step toward leveling the playing field. Some of the mathematics educators we queried were 
skeptical that the CCSSM would make a difference in that regard. Said one: 

I see almost nothing in the CCSSM that indicates any concern for or understanding of 
the reasons that we have achievement gaps in mathematics. In fact, the premise of 
CCSSM seems to be simply that expecting the same learning by cdl students at the same 
time will overcome the many factors that lead to differential achievement. I guess it is 
important to do research to find out whether that claim is borne out in practice. But it 
would be nice to have such studies probe more deeply to attempt to understand why 
achievement gaps persist even after CCSSM adoption (as they are almost certain to 
persist). 

Some respondents noted the potential irony if policies regarding student expectations and 
opportunities lead to implementation of the CCSSM in ways that actually have the opposite 
effect. For example, one person expressed concern that “tracking and ability grouping of 


Horizon Research, Inc. 


16 


November 2011 



students will grow rather than diminish.” Similarly, there was concern that student 
disengagement or disinterest in mathematics, failure, and dropout rates will increase over time 
because the standards will be unrealistically high and discourage students, and that the way 
decisions are made and resources allocated in implementing the standards will create “a barrier 
rather than a bridge for traditionally low performing subgroups.” 


Indicators of Alignment with the CCSSM 

As noted earlier, this project aimed to develop a priority research agenda to understand how the 
CCSSM are actually influencing mathematics education, whether that influence proceeds largely 
as the proponents intend, plays out as critics are concerned it might, or some combination of the 
two. In any event, research on the influence of a given set of standards depends on having a 
good sense of what elements of design and enactment that might vary can serve as indicators of 
what “counts” as aligned. As a key first step, a framework needs to be developed that describes 
the measurable features of materials, programs, and practices that can serve as indicators of 
alignment with the CCSSM. And then instruments need to be developed to measure those 
elements in valid and reliable ways in order to make determinations of the degree and nature of 
alignment with the CCSSM. 

The survey of mathematics educators revealed interesting differences in how “alignment” was 
defined. For example, in describing classroom instruction aligned with the CCSSM, some 
respondents focused on the salient features of these standards, including grade- specific content, 
standards progressions, and emphasis on mathematical practices. Others included features of 
classroom practice that are not mentioned in the standards documents, but may be important in 
order for students to be able to learn the intended content. We quickly recognized a need to 
distinguish between alignment and quality. In the end, we decided to highlight alignment 
indicators when investigating the extent of influence of the CCSSM on a component of the 
system; to include examples of quality indicators primarily when looking at impacts further 
downstream; and to encourage the research community to consider additional indicators of 
quality as appropriate in order to understand the relationship between extent of alignment with 
CCSSM and an outcome of interest. 

As an example, the following description of classroom practice focuses on alignment with the 
CCSSM, and does not include other features related to the quality of instruction that might be 
documented in order to understand why CCSSM-aligned instruction leads to the desired student 
outcomes in some cases but not others, e.g., patterns of classroom discourse. (See Appendix B 
for other sample indicators of alignment.) 
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Classroom Practice Aligned with the CCSSM 


In classroom practice aligned with the CCSSM, students are engaged with materials 
that address standards for mathematical content at the appropriate grade/course level, 
and teachers and students spend more time than previously on the areas focused on in 
the standards. The tasks that form the basis of instruction have the potential to develop 
students’ understanding (i.e., they are not all procedural) of the targeted standards, and 
the standards for mathematical practice are evident in the instructional delivery of 
those content standards. And while instruction focuses on grade-appropriate content 
and practice, the standards for mathematical content and practice that students engage 
with are clearly connected to prior standards and foreshadow later standards. Students 
are assessed both on their knowledge of the standards being addressed and on the 
conceptual linkages with the standards that came before. 


Clearly, research on the influence of the CCSSM needs to take into account factors other than 
alignment with the standards that have been shown to make a difference in effectiveness. In this 
example, classroom practice may be well aligned with the CCSSM in terms of content addressed, 
but teachers in some classrooms may not be able to monitor student understanding and adjust 
instruction accordingly, a potentially important mediating variable. Similarly, the content of a 
professional development program may be well aligned with the CCSSM, but have little impact 
on classroom practice if teachers are not given sufficient opportunity to consider the implications 
of what they are learning for their instruction. Identifying “quality indicators” for the various 
components of the mathematics education system addressed in the priority research agenda was 
well beyond our charge, but we strongly encourage researchers to consider quality, particularly 
when examining the relationship between extent of alignment in one component and effects 
further downstream. 

Consistent with the notion of common, higher standards for all students is the need to determine 
if the CCSSM are effective for students across contexts and demographic groups, so a priority 
research agenda needs to include a focus on who is being affected, how, and why. In addition, 
the timeline for implementation needs to be taken into account; “up close” studies of how the 
CCSSM influence (or fail to influence) the system, and why, in places where there is reasonable 
opportunity for them to do so should precede larger studies to determine the extent of influence 
“at scale.” 


Priority Lines of Inquiry 

All of the data collection activities in this project included some focus on priorities for research 
on the CCSSM, i.e., interviews with the developers, small group meetings of mathematics 
educators and policy researchers, and surveys of mathematics educators. Information from these 
initial data collection activities was synthesized and drawn on in the conference of mathematics 
education and policy researchers to identify lines of inquiry that would be important for 
understanding the influence of the CCSSM. 
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Project staff used information resulting from project data collection to identify 12 “priority lines 
of inquiry;” as can be seen in Appendix C, these included key leverage points in the mathematics 
education system as well as desired outcomes. 


Feedback on the Draft Report 

We thought it important to get feedback from individuals with diverse perspectives on the 
CCSSM and on education research. The draft report, which included an introductory section and 
the priority lines of inquiry, was sent for external review to individuals who had attended the 
conference; individuals who had not attended the conference but had provided input at earlier 
stages in the project; and individuals who had not been involved in generating the draft priority 
research agenda. The 39 people who reviewed the draft report had many suggestions, ranging 
from minor wording changes to major reorganization and streamlining of the document. The 
following section summarizes some of the key issues that were raised and how we have 
addressed them. 

Reviewers’ Feedback on Introductory Sections 

Most reviewers indicated that the overview of the CCSSM provided in the draft report was a fair 
and straightforward description. At the same time, several reviewers noted a need to “clean up 
the language” in the introduction as it related to progressions, emphasizing that standards 
progressions, learning progressions, and learning trajectories are not the same. Similarly, others 
noted the need to be clear about the origin of statements made in this section. Said one: 

Sometimes it is difficult to tell whether you are reporting your own opinions ( the authors 
of the report), summarizing the opinion expressed in common core documents, or 
reporting results of your interviews. 

Reviewers had a number of suggestions for refining the NRC framework described in the 
introduction: adding feedback arrows to avoid giving the impression that the process is a linear 
one, and adding to the list of contextual forces. Said two reviewers: 

No conceptual model has ever been created where someone couldn ’t say — well, what 
about this, and shouldn ’t the arrows be here and there also? So, in that vein... where 
does professional community and interaction/collaboration fit in the conceptual 
framework? Isn ’t interaction (state level, district and school as well) a main avenue by 
which the common core are supposed to work (and a dimension on which they are 
different from individual state standards? ) Certainly the idea of collaboration and 
professional community might fit into certain of the priority lines of inquiry, but I 
wondered about whether the channels of influence should include the economies of scale 
idea ? 

Might acknowledge that your diagram of influence makes the channels of influence 
appear more linear than they will be. It doesn ’t, for example include the feedback loop 
that is likely to change various items on the left based on student performance. I would 
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say that you have deliberately made this simplification as a strategy for the initial wave 
of research. 

Reviewers generally agreed that in exploring how the CCSSM exert an influence on mathematics 
education, the priority research agenda should focus both on the developers’ theory of action and 
on concerns that have been expressed. A number of reviewers called for a much more detailed 
theory of action to support the priorities in the research agenda. For example, referencing the 
page numbers in the draft report one reviewer wrote the following: 

I took seriously the goal of the project as producing “the kinds of evidence needed to 
determine and understand the effects of the CCSSM on mathematics education in the 
United States” (p. 3). I believe pretty strongly that if this is the gocd, one needs a quite 
well-developed “theory of action ” (p. 4). The framework provided is a beginning of such 
a theory (p. 4, and especially p.16). But, to collect data on the factors that might explain 
why the CCSSM is having particular effects, I believe you ’ll need a more developed 
theory. 

Similarly, in reacting to research questions about awareness and use of resources/tools, e.g., for 
selecting instructional materials, another reviewer wrote: 

These last two point to something I’d like to be a bit more explicit — a theory of action 
about how these resources are supposed to be useful/influential, and a study of the 
bottlenecks and positive points of leverage that help/hinder their effectiveness. 

Some reviewers appreciated the detailed description of our methodology, in some cases asking 
for additional information, while others found that like an overly lengthy warm-up act, it 
distracted from the “main event.” 

Revisions to Introductory Sections 

We revised the language on “progressions,” and tried to be clearer about the data sources. We 
decided to provide a brief summary of the methodology in the priority research agenda and a 
more detailed version in this technical report. Adding nuances to the NRC framework and 
describing the CCSSM theory of action in detail were well beyond our charge, not to mention 
our budget. We do note the importance of having a conceptual framework and focusing research 
based on a theory of action in the methodological considerations section. 

Reviewers’ Feedback on Lines of Inquiry 

Several reviewers commented positively on the fact that the research agenda was so 
comprehensive. At the same time, a number of reviewers suggested that having separate lines of 
inquiries obscured the need to understand the interrelationships among, for example, content- 
focused professional development, teacher knowledge of mathematics, how the CCSSM were 
addressed in classrooms, and student learning. 

Others noted a need to focus more research attention on the key features of the CCSSM, e.g., the 
extent to which having common state standards is in fact leading to increased collaboration 
among states; and on the potential unintended consequences described in the draft report. Still 
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others noted areas that needed more emphasis, most notably Race to the Top, state policy, 
mathematics faculty, technology, and teacher content knowledge. Finally, one reviewer noted an 
additional line of inquiry focusing on student out-of-school learning opportunities similar to 
those in the high performing countries whose standards were considered in developing the 
CCSSM. 

Quite a few reviewers indicated a need for a smaller number of priorities, or a “prioritization of 
the 12 priority lines of inquiry.” For example: 

The strongest overall impression I have is that your exhaustive treatment of issues and 
priority questions within each may be thoughtful and sensible, but with the sheer number 
of “priorities, ” the research agenda is too long to direct the field as well as it might. 

As would be expected, reviewers had somewhat different perspectives on the top priorities for 
research, in part because some were considering what was most urgent to study and others what 
was most important to study, even if the research could not begin for some time. Several 
reviewers noted that research on both classroom practice and student outcomes are essential, 
with one noting that these “are without question the most important and, if nothing else comes to 
pass, then at least study of these two are critical.” Another reviewer related the two, suggesting 
that “the key thing to study is whether a faithfully implemented CCSSM-aligned curriculum 
yields the hypothesized learning results.” 

Many reviewers highlighted the importance of research on the consortia assessments. Said one, 
“To my mind, establishing the validity of the assessments used to measure student learning is the 
absolute highest priority.” Said another: 

This is the big one and should be given top priority for research investments. 

Professional societies and funding agencies should be proactive in assuring that the 
questions about assessment are studied thoroughly, and this can 7 start too soon. 

Research on key “drivers” of implementation — not only the consortia assessments, but also 
curriculum materials and professional development — was considered important by many in order 
to understand the nature and extent to which the CCSSM are permeating the system. Others 
thought studies of how states and districts of varying characteristics went about transitioning to 
the CCSSM would be particularly important. 

Research in areas such as development of capacity to implement the CCSSM was of generally 
less interest to reviewers. For example, in relation to state, district, and school leader capacity, 
the draft report noted that: 

Research in this area will help the field understand (a) what knowledge, skills, and 
processes are important at various levels of the system in order to have high-quality 
implementation of these particular standards, and (b) how those capacities can be 
developed most effectively given limited time and resources. 
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A reviewer commented as follows: “Yes, of course, but there is little unique to CCSSM about 
these questions. This should be a low priority research investment.” 

A few reviewers suggested a need for more guidance on study design, noting for example, the 
utility of studying states whose previous standards were similar to the CCSSM, and states where 
the two sets of standards were quite different; or making comparisons between states that had 
and had not adopted the CCSSM. 

Several reviewers suggested incorporating more of what is known from prior research e.g., in 
relation to implementing standards, high quality professional development, and effective 
instruction. Others suggested referencing: (1) useful instruments, e.g., the Surveys of Enacted 
Curriculum (Council of Chief State School Officers (CCSSO) Surveys of Enacted Curriculum 
(SEC) Collaborative Project, 2005) for studying alignment between standards and curriculum 
materials; (2) recommendations for how research should be conducted, e.g., the NRC report on 
studying curricular effectiveness (National Research Council, 2004); and (3) recent 
recommendations about implementation/research in relation to the CCSSM, e.g., from the 
CSMC conference on curriculum design (Confrey & Krupa, 2010). 

A number of reviewers also indicated a need for more guidance on the timeline for research. 

They noted that there were some important questions that needed to be addressed right away — 
for example, how and why particular decisions were made in the development of the consortia 
assessments. In contrast, some important questions could not be addressed for quite some time, 
such as the impact of those assessments on teacher decisions. 

Revisions to Lines of Inquiry 

We reviewed the draft research agenda based on feedback to the various lines of inquiry, and to 
the suggestions that cut across lines of inquiry, such as the need to focus more attention on key 
features of the CCSSM. We decided to be more selective in the priority research agenda, for 
example, including “generic” issues such as capacity development in research to understand the 
effectiveness of efforts to implement the CCSSM, rather than as designated foci of research. We 
also decided to include the original lines of inquiry in this report, limiting edits to correcting 
factual errors, and including representative excerpts from the feedback received for each line of 
inquiry so interested readers could see the evolution of research foci from the draft to the final 
report. (A systematic search of the research literature/available instruments, and inclusion of the 
resulting references was well beyond our scope.) 

Finally, we decided to describe the priority research agenda in terms of “research approaches,” 
such as in-depth case studies to understand how and why changes were (or were not) occurring; 
studying the relationship between alignment of one part of the system and changes downstream; 
and looking at the status of alignment of parts of the system over time. We also tried to give 
some sense of when each of the suggested investigations should occur. Although we retained the 
section on “methodological considerations” from the draft report as it had been very well 
received by the reviewers, we did not recommend particular study designs within any of those 
approaches. 
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Priority Research Agenda 


The priority research agenda described below was developed with considerable input from 
mathematics education and policy researchers. (See Appendix D for a list of contributors.) It 
takes into account the defining features of the CCSSM; key leverage points in the system; both 
how the CCSSM are intended to influence the system and possible unintended consequences; 
and what can be learned from various types of research.” Another important consideration in 
designing research on the influence of the CCSSM is the strategic choice of mathematics content 
on which to focus. The CCSSM represent substantial shifts in some content areas, but are quite 
similar to prevailing practice in others. As a consequence, research on some content areas will 
be more telling than others regarding the influence of the CCSSM. Topics for which the primary 
change is that the CCSSM provide greater or lesser emphasis than is typically the case currently 
can be investigated to determine if corresponding changes in emphasis occur in various parts of 
the system. However, additional areas of “telling content” should be identified, such as where 
the mathematical treatment or progression of ideas is quite different in the CCSSM than is 
currently predominant. For these topics, qualitative changes in the mathematical approach, 
sequence, and connections would be important to investigate. The standards for mathematical 
practice should also be a focus of research because they are intended to be a central part of 
mathematics education across K-12 grade levels. 

We believe that implementing sets of studies based on this research agenda will generate 
knowledge that will be useful in improving the implementation of the CCSSM, and in improving 
both the design and implementation of future sets of standards. The suggested research priorities 
are approached through a variety of studies that, taken together, will provide both broad and deep 
knowledge about the influence of the CCSSM. First, the recommended priority research agenda 
includes case studies focused on system components that are expected to exert strong leverage, 
as well as case studies to look deeply into the decisions and actions of states and districts as they 
respond to the CCSSM, and the consequences. These studies can yield careful descriptions that 
offer insight into not just what influences are evident, but how and why those influences came 
about. Case studies are particularly well suited for these targets of research because they can be 
responsive to both anticipated and unanticipated developments, can examine potentially different 
points of view at play, and can attend to the role of context. 

Second, the proposed research agenda includes relational studies, initially proofs of concept, then 
broader studies of conditions of effectiveness, and finally experiments to determine whether 
interventions can produce conditions of effectiveness in various contexts. The purposes for these 
three types of studies are different and complementary. Proofs of concept are generally 
opportunistic, or conducted under fairly special circumstances; they can be used to establish the 
viability of a particular relationship. Broader studies of the conditions of effectiveness extend 


2 

“ It is important to note that while the priority research agenda provides recommendations for what is most 
important to study, it stops well short of study designs. It was beyond the scope of our project to do the extensive 
review of the literature necessary to describe what is already known in each of these key areas; determine how 
current understanding needs to be taken into account in designing research studies; specify indicators that 
components of the system being investigated are or are not aligned with the CCSSM; identify instruments that could 
be adapted for use in research on the influence of the CCSSM, etc. 
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beyond those special circumstances to examine the range of conditions under which particular 
relationships exist, and for whom. These studies can also offer explanations of why and how 
particular conditions result in various outcomes. Finally, experiments/quasi-experiments are 
useful for establishing whether creating particular conditions in fact facilitates the relationships 
of interest, and for whom. 

Third, the priority research agenda includes status studies, some that involve secondary analysis 
of data available through on-going data collection efforts; others that piggyback onto existing 
efforts by adding a focus on the CCSSM; and some newly-crafted endeavors specific to the 
CCSSM. Status studies are particularly useful for taking broad snapshots of indicators of the 
health and quality of the mathematics education system at a given point in time, and for 
examining variations over time. They can be conducted on large, representative samples, 
lending themselves to disaggregation in order to investigate status and trends within subgroups 
and specific contexts of interest, and to compare status and trends across subgroups and contexts. 

For a number of questions throughout the research agenda, we emphasize the importance of 
attending to various subgroups of students who are educated in different contexts, to underscore 
the importance of disaggregating results, attending to potentially differential findings, and 
conducting targeted studies to investigate questions regarding equity of opportunity and 
outcomes. The term “various subgroups of students” is intended to include variations in 
socioeconomic backgrounds, gender, race/ethnicity/culture, language, immigration status, 
disability state, and educational needs. It is also intended to highlight the need for studies to 
investigate opportunities and outcomes for students whose academic achievement of the CCSSM 
lags behind, meets, or exceeds expectations. Similarly, the phrase “educated in different 
contexts” is intended to emphasize the need to attend to, and target studies to, varying state, 
district, community and family, school, and classroom (including virtual and distance learning) 
settings. 

To support cumulation of knowledge, research efforts drawing on any of these approaches must 
be conducted with attention to quality and rigor in design, conduct, and reporting. Cumulating 
knowledge also depends on establishing shared definitions for some key terms, so that 
differences in meaning for the same words, or the use of different words to represent the same 
ideas, do not obscure real points of agreement or disagreement. Along with the priorities for 
research, this report offers a set of methodological considerations, both to strengthen the 
evidentiary basis for the claims and conclusions of individual research studies, and to enrich the 
broader enterprise of research on the influence of the CCSSM. 

The priority research studies are organized and described below by study type. This choice of 
organization was useful for providing the rationale and specifying research questions for each 
recommended focus of research. This organization, though, is not meant to imply prioritization 
of study types. Further prioritization within these recommended sets of studies should be 
considered in terms of when particular studies are likely to be most informative, and what 
aspects are most critical to understand about the response of the system, the resulting outcomes, 
and evidence to improve the standards. We present a brief discussion of these considerations, 
organized by the three overarching questions presented in the introduction, followed by further 
explication of the recommended studies. Within each study type, the most essential priorities are 
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listed first and indicated with an asterisk (*), followed by other high priority studies. Setting 
priorities, by its nature, requires difficult choices. Additional areas for research that are both 
interesting and important are included in Appendix E. 

• How is the mathematics education system responding to the introduction of the CCSSM? 

Efforts to implement the CCSSM began as soon as they were released; publishers began revising 
instructional materials, states and districts began developing transition plans, and a host of 
players began offering professional development to teachers. Although studying any of these or 
other efforts would provide information about how the system is responding to the CCSSM, in 
establishing a priority research agenda, we took into account existing research on which 
components of the system exert the most leverage on K-12 education. In particular, we 
identified the development and influence of the consortia assessments, and the development and 
selection of curriculum materials, as the most important targets for research on how the system is 
responding to the CCSSM. Here we include examining the extent to which having common 
standards has led to greater collaboration and economies of scale, as the common core theory of 
action predicts. 

Also important, but of somewhat lower priority, are case studies of the transition process, as 
identifying strategies that appear to be effective across a variety of state and district contexts will 
help inform the implementation of the CCSSM and future sets of standards. Similarly, as 
curriculum materials are developed and revised to reflect the CCSSM, it will be important to 
analyze the most commonly used curriculum materials in terms of alignment with the standards 
and with the assessment frameworks. 

There is no question that research in other key areas would provide a fuller picture of the 
influence of the CCSSM, as well as help the field understand the conditions under which these 
standards are and are not leading to the desired outcomes. Among the areas of interest, but 
considered lower priority for research are how other important components of the mathematics 
education system are responding to the CCSSM, including pre-service and alternative 
certification programs; professional development programs; institutions of higher education; and 
professional societies. (See Appendix E.) 

• What happens, and for whom, as a result? 

Research may determine that the mathematics education system is in fact changing to align with 
the CCSSM, but that is only part of the story. Early studies of impact on students in places 
where instruction is aligned with the CCSSM will provide important information, but 
effectiveness with early adopters may not predict broader impact. Research on a larger and more 
representative sample of classrooms, using a variety of outcome measures, will be needed to 
identify the conditions under which the CCSSM are effective, and for whom. Similarly, while it 
will take time for the earliest cohort of high school graduates to have experienced the CCSSM 
throughout their K-12 education, given the goal of preparing students for college/careers, 
research in this area is essential for testing the theory of action of the CCSSM. The key question 
is whether students whose K-12 learning opportunities and outcomes reflect the CCSSM are in 
fact well prepared for post- secondary education and careers. And over time, research on the 
extent to which there are improvements at scale on college and career readiness, as well as on 
key national and international K-12 assessments, will be an extremely high priority. 
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Case studies of how teachers interpret the CCSSM and the implications they see for their 
instruction, are also included in the priority research agenda, but are considered of somewhat 
lower priority. Similarly, periodic status studies of representative samples of teachers and their 
teaching as they relate to the CCSSM would be helpful in assessing the influence of the 
standards, but are not deemed as high in priority as studies that explore the relationships between 
implementation of the CCSSM and student outcomes. 

• How can the CCSSM and future standards be improved? 

The priority research agenda includes studies that can inform revisions to the CCSSM, as well as 
development of future sets of standards. Included in this category are investigations of standards 
progressions that researchers believe are not adequately supported by prior research and/or where 
alternative progressions may be more effective. Similarly, studies to establish learning 
progressions for the standards for mathematical practice would be very informative, as little is 
known in this area. 


A. Case Studies 

The large number of states adopting the CCSSM, and the major efforts underway to support their 
implementation, suggest a need for research to begin immediately to document the process, 
including the rationale for, and consequences of, the various actions being taken. The specific 
targets for case studies that emerged as priorities are the cross-state assessment consortia; efforts 
to revise/develop curriculum materials in response to the CCSSM; state/district responses to the 
CCSSM; and, as efforts to introduce and implement these standards proceed, teachers’ responses 
to the CCSSM. 

* Priority Case Study Focus #1: Development of the consortia assessments * 

The two cross-state assessment consortia funded by the U. S. Department of Education are 
viewed as a priority for study because high-stakes assessments are known to be a major force 
influencing the nature and content of the curriculum students will experience. In addition, they 
represent a key domain in which the commonality of standards supports collaboration across 
states. How content ideas and practices are treated on the assessments will send strong 
messages, whether intended or not, about which content within the standards is most important. 
Consequently, it will be critical to understand how and how effectively the development of 
consortia assessments guards against an unintended “narrowing” of the curriculum. We suggest 
that case studies of the development of these assessments be carried out by “embedded 
researchers,” who can document the process, describe the trade-offs that are considered, and 
convey the rationale for the various decisions. These case studies should focus on the following 
questions: 

1. How are the CCSSM standards for mathematical content and mathematical practice, 
as well as the standards progressions, being interpreted by key decision-makers in the 
assessment consortia? What specifications for assessment design result from these 
interpretations? Do the tests strongly focus where the standards strongly focus? 
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2. How do various affordances and constraints influence the nature of the assessments: 
the purposes for which the assessments are being developed; current state of the field 
in assessment development; potential opportunities and benefits, as well as costs and 
constraints, of technology-enabled administration of assessments; and the diversity of 
contexts of intended use, e.g., with students whose native language is other than 
English, with students who have varying access to technology? 

3. In what ways and to what extent are the capacities and resources of collaborating 
states and other entities being leveraged in the work of the assessment consortia? 
What affordances and challenges arise in these efforts due to cross-state 
collaboration? 

4. What other resources are the assessment consortia producing, and for what purposes? 
For any resources distributed prior to the release of the assessments themselves, what 
influences are these resources having on states and other parties that are accessing 
them? 

* Priority Case Study Focus #. 2 : Influences of the consortia assessments* 

In 2014-15, the assessment consortia are due to release their products for use in state assessment 
and accountability systems. At that point, it is likely that the influence of the CCSSM will be 
strongly mediated by the assessments. We suggest that case studies conducted over the three- to 
five-year period after the release of the consortia assessments address the following questions: 

1. How are states intending to use the assessments for accountability and decision- 
making purposes? Over time, what information do the assessments provide for these 
purposes? What are the accountability consequences? What decisions are informed 
by the assessments, with what consequences, for whom? 

2. How are districts and schools planning to use the assessments for their accountability 
and decision-making purposes, and for formative and diagnostic purposes? Over 
time, what information do the assessments provide for these purposes? What are the 
accountability consequences? What decisions are informed by the assessments, with 
what consequences, for whom? 

3. In what ways are curriculum development efforts and teacher development efforts 
influenced by the portrayal in the consortia assessments of the standards for 
mathematical content and practice, and the standards progressions, as well as by how 
the assessment results are being used by states, districts, and schools? 

4. In what ways is the enacted curriculum influenced by the portrayal in the consortia 
assessments of the standards for mathematical content and practice, and the standards 
progressions, as well as by how the assessment results are being used by states, 
districts, and schools? 


Horizon Research, Inc. 


27 


November 2011 



* Priority Case Study Focus # 3 : Curriculum materials development/revision process * 
Textbooks/programs are a key leverage point in the system, as how they organize and present 
content exert a strong influence on the curriculum that students experience. Given the enormous 
market for curriculum materials that address the content of the CCSSM, publishers are already 
responding to the CCSSM. Consequently, understanding how and why curriculum materials are 
or are not developed/revised in response to the CCSSM is a priority for research. 

Since some states are moving ahead quickly in implementing the CCSSM, it is likely that major 
curriculum materials publishers will respond by revising existing materials. Revisions may take 
various forms — e.g., moving topics into particular grade levels, re-sequencing topics within 
grade levels, modifying the mathematical approach for particular topics, and adding specific ties 
to the standards for mathematical practice. And there will also be efforts to develop new 
materials specifically aimed at the CCSSM. 

In studying the curriculum development/revision process, it will be important to include a variety 
of materials (e.g., at the high school level those that are course- specific, and those that take an 
integrated approach), and to be strategic in the selection of mathematics content areas on which 
to focus. Choosing content topics that vary from the currently predominant approaches will be 
particularly telling, such as new content topics, or content topics that are approached, sequenced, 
or connected to other topics differently. Case studies of the revision/development process should 
address the following questions: 

1. How are the CCSSM standards for mathematical content and mathematical practice, 
and the standards progressions, being interpreted by curriculum materials developers? 
What plans for revisions result from these interpretations? Do new or revised 
curricula focus strongly where the standards focus strongly? How are developers 
planning to address variations across states introduced by the option of including 15 
percent state- specific standards? 

2. What expertise and resources are developers using to inform their revisions to 
curriculum materials? 

3. To what extent does the development/revision process take into account the needs of 
various subgroups of students and contexts of schooling? 

4. What consideration is given in the development/revision process to providing 
enrichment/remediation opportunities for students whose progress differs from the 
expectations laid out in the CCSSM? 

5. In what ways are curriculum developers revising the “educative” components in their 
materials intended to inform and provide professional learning opportunities to 
address the needs of teachers as they use the materials? 

Some mathematics educators have expressed concern that widespread adoption of the CCSSM 
will lead to lack of innovation in curriculum materials, and in turn, diminished opportunity for 
the field to continue to learn about effective organization and sequencing of mathematics content 
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for student learning. Similarly, there is concern that the CCSSM do not emphasize the use of 
technology for learning and doing mathematics to the extent necessary for college/career 
readiness. These kinds of concerns suggest a need for case study research to look across cases of 
the most widely used materials aimed at the CCSSM, and to compare them with efforts that are 
not attempting to address the CCSSM, addressing additional questions, including: 

6. What similarities and differences in content organization and presentation exist across 
curriculum materials intended to address the CCSSM? What accounts for these 
similarities and differences? 

7. When addressing the CCSSM, what opportunities, if any, do curriculum materials 
developers have for innovation, including the integration of technology? What 
constraints, if any, restrict innovation? 

8. When addressing mathematics content not included in the CCSSM, what 
opportunities, if any, do curriculum materials developers exercise opportunities for 
innovation, including the integration of technology? To what extent is the inclusion 
of state- specific standards (the 15 percent allowance) providing opportunities for 
innovation? What constraints, if any, restrict innovation? 

Priority Case Study Focus #4: State/district responses to the CCSSM 

Although 40+ states have adopted the same set of mathematics standards, individual states and 
the districts within them face very different challenges in transitioning to full implementation of 
the CCSSM. Studying how and why states/districts are responding to the CCSSM (including 
choosing not to adopt them) is an important part of understanding their influence. The proposed 
research agenda suggests case studies of states and districts, including a set of locales that vary in 
demographics; policy context; commitment to the CCSSM; and history with content standards, 
including how similar the CCSSM are to their most recent standards. 

Some states and districts have made a strong and immediate commitment to implement the 
CCSSM at scale, while others intend to phase in the CCSSM over time. In addition to 
differences in phases-in approaches, there are differences in perspectives regarding how much 
change the CCSSM represent from current standards. The authors of the standards have 
indicated that the CCSSM convey a major shift in the focus of mathematics education in the 
United States. However, there are signs that some states, some observers, and some sectors in 
the education system see the implementation of the CCSSM as entailing only minor changes. In 
case studies of state/district transitions will be critical, considering both content that appears 
fairly similar and content that seems quite different from previous standards, as well as both 
mathematical practices and standards progressions. Additionally, it will be important to 
understand the rationale for any shifts in organization at the high school level (e.g., from course- 
specific to integrated pathways, or vice versa) in light of the CCSSM. 

Different states will have widely varying resources and incentives to support implementation of 
the CCSSM, federal Race to the Top awards for some states and not others being a primary 
example. Across states and districts, the existing leadership and capacity, and the ability to 
develop leadership and capacity will vary considerably, likely leading to differences in how, and 
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how well, they are able to implement the CCSSM. Case studies of states and districts in 
transition, ideally conducted by researchers embedded in the process, will provide a great deal of 
information about how the CCSSM are influencing mathematics education. 

a. Initial Case Studies of Adopting States/Districts 

Initially, the following questions should be addressed in a variety of jurisdictions which are 
planning to implement the CCSSM: 

1. What policy levers (e.g., mandates, incentives) are states using to influence which 
parts of the system (e.g., curriculum, teacher development, assessment) and at what 
level (state, local, classroom)? How do the states differ, and why are they taking 
different paths? 

2. How do broader opportunities and constraints, in particular cross-state collaborations, 
cross-district collaborations, and federal initiatives, influence plans for supporting 
implementation of the CCSSM? 

3. How are states/districts modifying policies and programs to support implementation 
of the CCSSM across the range of contexts they include? 

4. How are key stakeholders (policy makers, leaders in business and industry, higher 
education, non-govemmental organizations, teacher unions, parents, and the general 
public) exerting influence on plans for CCSSM implementation? 

b. On-going Case Studies of Adopting States/Districts 

Over time, following the transition to the CCSSM in these selected places, and in others that may 
emerge as experiencing particular successes or challenges, case studies will remain among the 
priorities for research, addressing the following questions: 

1. How do specific policies, programs, and resources intended to support 
implementation of the CCSSM play out? How do successes and challenges of 
implementation, in turn, influence these policies, programs, and resources? 

2. What other policies, programs, resources, and contextual factors influence 
implementation of the CCSSM, and in what ways? 

3. How do broader opportunities and constraints, in particular cross-state collaborations, 
cross-district collaborations, federal initiatives, and programs and tools offered by 
third-party entities influence implementation of the CCSSM over time? 

4. How are key stakeholders (policy makers, leaders in business and industry, higher 
education, non-govemmental organizations, teacher unions, parents, and the general 
public) exerting influence on CCSSM implementation over time? 

5. What organizational systems are supporting implementation of the CCSSM? Which 
players are involved in implementation of the CCSSM, and how do they compare to 


Horizon Research, Inc. 


30 


November 2011 



the players involved in mathematics education improvement efforts prior to the 
CCSSM? 


6. What is the depth and breadth of penetration of the CCSSM within the state/district 
system? What parts of mathematics education in the state/district are affected and in 
what ways? What parts are not affected? With what consequences? 

7. Within the state/district, what variations in implementation of the CCSSM are 
evident? What accounts for those variations? For whom do these variations have 
important consequences? 

8. How are districts, schools, and teachers (a) helping students catch up when their 
progress falls behind the expectations of the CCSSM; and (b) providing enrichment 
opportunities for students who are performing above expectations? 

9. How are states/districts/schools responding to challenges associated with more 
rigorous standards, e.g., increased drop-out rates, low passing rates for particular 
subgroups of students on consequential assessments? 

10. What strategies appear to be effective across a variety of state and district contexts? 
c. Case Studies of Non-adopting States 

Although the CCSSM have been adopted in the great majority of states, there are states that have 
chosen not to adopt, and the possibility remains that some states that initially adopted the 
CCSSM may change their commitment over time. In understanding the influence of the 
CCSSM, how mathematics education evolves in non-adopting states provide an important point 
of contrast. At the same time, the CCSSM are anticipated to influence the mathematics 
education system nationally in ways that are likely to have important effects even in non- 
adopting states. Key questions to address in case studies of selected non-adopting states are: 

1. Why did the state choose not to adopt the CCSSM? 

2. How do the state’s mathematics standards compare to the CCSSM? What similarities 
and differences between the state’s standards and the CCSSM are considered 
important and why? 

3. What policies, programs, and resources within the state are intended to support 
mathematics education improvement in the state? How do these play out? 

4. How are key stakeholders (policy makers, leaders in business and industry, higher 
education, non-govemmental organizations, teacher unions, parents, and the general 
public) exerting influence on improvement efforts over time? 

5. What broader influences of the CCSSM initiative affect non-adopting states, and in 
what ways? 
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Priority Case Study Focus # 5 : Teacher responses to the CCSSM 
Unless the influence of the CCSSM ultimately reaches teachers of mathematics, any other 
influences will mean little for most students’ learning opportunities. Since teachers’ knowledge, 
interpretations, self-efficacy, beliefs, dispositions, and skill, as well as their specific intentions 
and plans, affect what transpires in classrooms, it is critical to understand how teachers respond 
to the CCSSM, and what kinds of classroom learning opportunities for their students result. At 
the same time, it is important to recognize that teachers are not likely to be influenced directly by 
standards, but rather by a combination of many sources that translate and communicate 
implications, intended or not, of the standards. Case studies of teachers’ responses to the 
CCSSM should address the questions below, ensuring that in aggregate these studies represent 
teachers of varying backgrounds and characteristics; working in a variety of school contexts and 
serving different student subgroups, with varying levels of collective effort in relation to 
implementing the CCSSM. 

1. What opportunities do teachers have to learn about the CCSSM and their 
implementation? What messages do teachers take from these opportunities? What do 
teachers see as the implications for their further professional learning? 

2. What implications do teachers see for their mathematics instruction? What aspects of 
their mathematics instruction do they see as validated by the CCSSM, and what aspects 
do they consider in need of change based on the CCSSM? Do teachers view these 
changes as positive, negative, some mixture of the two, or the latest in a long line of 
fleeting “reforms”? Why? 

3. What curriculum materials and resources, including textbooks, supplemental materials, 
assessment tools, and instructional technologies do teachers use to implement changes in 
their mathematics teaching in relation to the CCSSM? Why do they use these particular 
materials/resources, and with what effects? What available materials/resources do 
teachers not use and why? 

4. Over time, to what extent and in what ways do teachers perceive their practice aligning 
with the expectations of the CCSSM standards for content and practice, and the standards 
progressions? How do observers’ views of alignment compare and contrast with 
teachers’ perceptions? 


B. Relational Studies 

If standards are to make a difference in students’ opportunities to learn important and useful 
mathematics, various parts of the system need to align with those standards and work in concert. 
Research on the influence of the CCSSM requires studies to understand the extent to which 
implementation efforts result in alignment/non-alignment throughout the system, and the 
downstream consequences of those variations in alignment. 

Three types of relational studies are recommended for research to understand the influence of the 
CCSSM: (1) proofs of concept, (2) broader studies of conditions of effectiveness, and (3) 
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experiments to determine whether interventions can produce conditions of effectiveness in 
various contexts. The purposes for these three types of studies are different and complementary. 
Proofs of concept are generally opportunistic, or conducted under fairly special circumstances, to 
establish the viability of a particular relationship. Broader studies of the conditions of 
effectiveness extend beyond special circumstances to examine the conditions under which 
particular relationships exist, and for whom. These studies can also offer explanations of why 
and how particular conditions result in various outcomes. Finally, experiments, or quasi- 
experiments, are useful for establishing whether creating particular conditions in fact produces 
the outcomes of interest, and provides for further study regarding for whom these relationships 
hold. Studies of each type are recommended to address several critical relationships. 

* Priority Relational Study Focus # 1 : Relationship between the enacted curriculum and K-12 
student outcomes* 

Studies of the relationship between the classroom enactment of learning experiences aligned with 
the CCSSM and students’ achievement of the designated learning goals are a key priority in the 
proposed research agenda. A body of studies would need to be conducted to examine this 
relationship at different grade levels; for different standards for mathematical content and 
practice; with students of various backgrounds; and with students who experience different types 
of programs (e.g., course- specific and integrated mathematics programs, programs with varying 
degrees of tracking). In particular, it will be important to distinguish between persistent 
differences in achievement due to different instructional experiences, versus differences in 
achievement due to differential effectiveness of the same instructional experiences for various 
groups of students. In addition, this body of studies would collectively need to address a range 
of student outcomes of importance, including not only achievement of the standards for 
mathematical content and practice, but also dispositions toward the study and use of 
mathematics; and interest in, and aspirations for, STEM careers. These studies should address 
the following questions, attending to possible differences across various student subgroups and 
contexts: 

1. What outcomes result when the enacted curriculum students experience is closely 
aligned with the CCSSM? 

2. What outcomes result when the enacted curriculum students experience is aligned 
with the CCSSM in various ways and to varying degrees? How can these outcomes 
be explained by students’ classroom learning experiences? 

3. What instructional and contextual factors mediate relationships between alignment of 
the enacted curriculum with the CCSSM and student outcomes? How and why do 
these factors enhance or inhibit student outcomes? 

4. What student factors, including experiences they have outside of the school 
environment, mediate relationships between alignment of the enacted curriculum with 
the CCSSM and student outcomes? How and why do these factors influence student 
outcomes? 
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5. To what extent do students whose progress in achieving the CCSSM falls behind 
expectations have remediation and re-entry opportunities that allow them to attain the 
learning expectations of CCSSM in order to enter post-secondary education, careers, 
or career preparation successfully? What student factors mediate this relationship? 
What accounts for variability in the effectiveness of remediation and re-entry 
opportunities? 

6. To what extent do students whose progress exceeds the expectations of the CCSSM 
have enrichment opportunities that allow them to pursue mathematics study in greater 
depth and breadth? What school, teacher, and student factors mediate this 
relationship? What accounts for variability in the effectiveness of enrichment 
opportunities? 

* Priority Relational Study Focus #. 2 : Relationship among the enacted K-12 curriculum, the 
achieved K-12 curriculum, and college/career readiness * 

As noted earlier, the CCSSM are intended to specify the mathematical content understandings 
and practices considered fundamental for K-12 students’ preparation for college and careers. 
Studies addressing this relationship should be a priority once assessments are in place to 
document students’ achievement of the learning expectations of the CCSSM and sufficient time 
has passed for students completing their K-12 education to have experienced mathematics 
programs influenced by the CCSSM. 

Studies of the relationship of the enacted and achieved K-12 curriculum and students’ college 
and career readiness should also account for other aspects of readiness that might be affected by 
implementation of the CCSSM, such as readiness to use various technologies, and to leam new 
technologies, that are important for further education and career preparation and success. 

Priority studies of the relationship between students’ K-12 learning experiences, achievement of 
the CCSSM learning expectations, and post- secondary outcomes should address the following 
questions, in each case attending to possible differences across student subgroups and contexts: 

1. To what extent do students whose K-12 learning opportunities and outcomes indicate 
attainment of the learning expectations of the CCSSM (a) enter post-secondary 
education ready for credit-bearing mathematics and other STEM coursework, (b) 
successfully enter careers? What school, teacher, and other student factors mediate 
these relationships? What contextual factors of post-secondary institutions mediate 
these relationships? 

2. How do students whose K-12 learning opportunities and outcomes reflect differences 
from the learning expectations of the CCSSM fare in college mathematics and other 
STEM coursework, or careers requiring mathematical knowledge? 

Priority Relational Study Focus #3: Relationship between consortia assessment results and 
other measures, i.e., validation of consortia assessments 

Historically, student assessments, especially when they have high stakes attached, are a major 
lever in determining the content that is emphasized in K-12 mathematics education. The 
assessments produced by the two cross- state consortia are anticipated to be primary sources of 
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information about students’ attainment of the CCSSM, informing judgments and decisions about 
students, teachers, schools, programs, and policies. Consequently it is important to investigate 
the validity of expected and actual uses of results from the consortia assessments. 

These validation studies are considered relational because they will examine the relationship 
between results of consortia assessments (and the judgments and decisions that are made based 
on those results,) and results obtained from other measures, including other desired student 
outcomes that are not measured on the consortia assessments. First, studies are needed to 
determine whether the consortia assessment results are consistent with results obtained from 
more in-depth measures than are possible to administer on a very large scale. Second, results on 
long-standing measures, such as NAEP, should be compared to the results of the consortia 
assessments to determine if trends shown on the consortia assessments are seen in these other 
measures as well. Finally, investigations are needed of the extent to which results on the 
consortia assessments predict readiness for further study of K-12 mathematics, and for 
college/careers. Studies are recommended to answer the following questions: 

1. Throughout the K-12 grades, how do the consortia assessment results compare to 
results from more in-depth measures of students’ progress in achieving the standards 
for mathematical content and practice? Are formative and summative judgments 
based on the consortia assessment results supported by results of more in-depth 
measures? How, if at all, do these comparisons differ across subgroups of students 
and students educated in different contexts? 

2. How do trends over time on the consortia assessments compare to established 
measures of mathematics achievement, such as the National Assessment of 
Educational Progress? 

3. How does the content of high school assessments compare to higher education STEM 
faculties’ expectations for college readiness and employers’ expectations for career 
readiness? At high school graduation, how do the consortia assessment results 
compare to other measures of students’ achievement of the standards for 
mathematical content and practice? Are judgments based on the consortia assessment 
results (e.g., remedial vs. credit-bearing college course placement, qualification for 
careers) supported by results from more in-depth measures? How, if at all, do these 
comparisons differ across subgroups of students and students educated in different 
kinds of contexts? 

4. How are the consortia assessment results being used in evaluating teacher 
performance or progress, school status or progress, and program or policy 
effectiveness? Are judgments based on the consortia assessments supported by other 
evidence? 

Priority Relational Study Focus # 4 : Influences on the enacted curriculum 
The intent of the CCSSM is to ensure that all students in the United States, regardless of where 
they happen to go to school, have the opportunity to engage with a core set of key mathematics 
ideas and practices, sequenced and connected in a meaningful way. The CCSSM do not 
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prescribe how to go about enacting classroom learning experiences, or how to go about ensuring 
that the standards are enacted at scale in classrooms. The enacted mathematics curriculum, made 
up of elements described previously, results from the interplay of a wide range of teacher factors, 
teacher professional learning opportunities, and school factors, combined with available 
curriculum materials and resources, and assessment tools. Student characteristics, too, play a 
key role in constructing the enacted curriculum. 

Studies of these influences on the enacted curriculum should be conducted across grade levels, 
and in a wide range of contexts. Additionally, they should examine research-established aspects 
of classroom learning experiences that may not be addressed by the CCSSM, but are known to 
relate to student learning. To gain a comprehensive picture of the enacted curriculum, these 
studies must also consider a variety of mathematical content, standards progressions, and 
mathematical practices. Results from these studies would shed light on the kinds of learning 
opportunities students are encountering, aligned or not aligned with the CCSSM, and would 
provide evidence to explain variations. Research should address the following questions: 

1. How do state and district policies, programs, resources and contexts affect the enacted 
curriculum in classrooms? 

2. How do characteristics of curriculum materials, including both student materials and 
materials providing guidance and educative features for teachers, affect the enacted 
curriculum? 

3. How do teacher background factors (e.g., knowledge, skills, beliefs, dispositions), 
interpretations of the CCSSM, and expectations for student learning more generally, 
affect the enacted curriculum in their classrooms? 

4. How do teachers’ opportunities to learn about (a) the CCSSM, and (b) how to address 
those expectations, affect the enacted curriculum in their classrooms? 

5. How, if at all, does the enacted curriculum that students experience differ across 
various subgroups of students and in different contexts? What teacher background, 
teacher opportunity, and school factors explain these variations? 


C. Status Studies 

The CCSSM are intended to improve the quality of mathematics education at scale. 
Consequently, understanding the influence of the CCSSM needs to include studies of the extent 
to which the system is improving, and for whom. The status studies that have been identified as 
priorities provide a picture of key aspects of the system. They would further allow for 
investigation of trends over time, and disaggregation so that comparisons can be made across 
subgroups and contexts. 

Several types of status studies are recommended to shed light on the influence of the CCSSM. 
First, the United States already invests in large-scale, representative data collection efforts (e.g., 
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National Assessment of Educational Progress) that can yield information to suggest ways that the 
CCSSM may, or may not, be having an impact on the mathematics education system nationally. 
Second, since these large data collection efforts may not have historically addressed certain 
indicators of interest, e.g., in relation to the standards for mathematical practice, items could be 
piggy-backed onto existing efforts to collect these data. Third, for some areas of interest, such as 
the prevalence and quality with which the CCSSM standards progressions are addressed in 
curriculum materials or in classrooms, there may not be existing national efforts that lend 
themselves to the needed data collection, so new efforts would be necessary. 

* Priority Status Study Focus # 1 : K-12 students’ interests, aspirations, and achievement * 

Many stakeholders will consider student achievement outcomes the bottom line forjudging the 
influence of the CCSSM. Undoubtedly, the results of consortia assessments or alternatives that 
states employ for standardized testing will receive considerable attention. Also, U.S. students’ 
performance on national assessments (e.g. National Assessments of Educational Progress 
(NAEP)) and international assessments (e.g.. Trends in Mathematics and Science Study (TIMSS) 
and the Programme for International Student Assessment (PISA)), will remain important for 
gauging progress in student preparation over time and in relation to other nations. It is not clear, 
as yet, that any of these assessments will provide evidence regarding students’ success or 
struggles with specific features of the CCSSM, such as advancement along the standards 
progressions or growth in understanding and use of mathematical practices; other assessments 
may be needed to conduct status studies of these key features. 

In addition to achievement, a number of other student outcomes are proposed as priorities for 
status studies throughout the K-12 grades, including dispositions toward the study and use of 
mathematics, confidence in mathematics knowledge and abilities, and interest in mathematics 
and other STEM fields. Advanced course taking and success have also been important metrics 
forjudging the quality of mathematics education, so we recommend measuring enrollment in 
Advanced Placement, advanced International Baccalaureate, and dual enrollment college-level 
mathematics as the implementation of the CCSSM proceeds. 

A very strong priority for investigating the status of student outcomes is to examine results for 
different subgroups of students and different contexts of schooling. It is also important to 
disaggregate results by grade level and content domains, as students’ success and challenges are 
likely to vary along these dimensions. Key research questions in relation to K-12 student 
outcomes are: 

1. Across grades K-12, to what extent are students achieving various levels of mastery 
of the standards for mathematical content and practice? What variations in 
achievement are evident across student groups and contexts of schooling? 

2. How many and which students are enrolling in, and succeeding in, advanced 
mathematics courses in high school? To what extent are students who are prepared 
for study in advanced mathematics courses either not enrolling, or not succeeding, in 
these courses? 
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3. What is the nature and distribution of students’ dispositions toward mathematics; 
confidence in their mathematics abilities; and interest in, and aspirations for, STEM 
study and careers? What differences are evident for various student 
subgroups/contexts? 

* Priority Status Study Focus #. 2 : College/career readiness * 

The CCSSM describe the mathematical content understandings and practices that have been 
proposed as key to K-12 students’ preparation for success in college and careers. Over time, one 
measure of success of the CCSSM initiative will be evidence that high school graduates are in 
fact increasingly becoming prepared for the mathematical demands of their post-secondary 
education and work. And in particular, there is widespread interest in the STEM “pipeline,” 
ensuring that a larger and broader pool of students have interest in, and preparation for, careers in 
STEM fields. Status studies of students’ college and career readiness should address the 
following questions, in each case including investigating variations across subgroups of students 
from varying K-12 educational contexts: 

1. To what extent are students entering college prepared for placement and success in 
credit-bearing mathematics courses? 

2. To what extent are students in college pursuing and succeeding in mathematics and 
other STEM majors? 

3. To what extent are students entering careers prepared for the mathematics demands of 
their work? 

Priority Status Study Focus #3: Analysis of curriculum materials and accompanying guidance 
for teachers 

As noted earlier, mathematics curriculum materials exert a strong influence on K-12 students’ 
learning experiences. A wide variety of curriculum materials have historically been available in 
the United States, including those that were developed initially to reflect the vision of the NCTM 
standards (NCTM 1989, 2000), and they have organized and presented content in substantially 
different ways. Curriculum materials developers have had little time to prepare for widespread 
adoption of the CCSSM, and they are likely to have different interpretations of the implications 
for particular content areas. Consequently, revisions of existing curriculum materials and the 
accompanying guidance for teachers are likely to result in variable alignment with the CCSSM 
both within and across curriculum materials. New materials will be developed over time and, 
given market forces, it is likely the CCSSM will have a strong influence on these materials as 
well. Monitoring the extent and nature of the alignment of a variety of curriculum materials with 
the CCSSM, and with the consortia assessment frameworks and tests, will be revealing regarding 
the potential reach of the standards by addressing the following questions for a variety of 
materials, including the most commonly used supplemental or replacement materials that may be 
used in place of, or alongside, full curriculum materials programs: 

1. To what extent are the standards for mathematical content at various grade levels 
evident in the materials, and how well do they reflect the intended rigor and depth of 
the CCSSM? 
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2. To what extent does the approach to, and sequence of, content in the materials reflect 
the standards progressions of the CCSSM? 

3. How and to what extent are the standards for mathematical practice evident in the 
materials, and integrated with the standards for mathematical content? 

4. To what extent are materials including content addressed by individual states as part 
of their 15 percent allowance? 

5. How much and what kinds of support are provided for teachers about how to address 
the key features of the CCSSM, and in particular how to assist students who are 
having difficulty with achieving the grade- specific or high school level standards, as 
well as those who would benefit from enrichment? 

Priority Status Study Focus #4: Alignment of the enacted curriculum with the CCSSM 
The ultimate aim of the CCSSM is to improve student outcomes, and it is through the learning 
experiences enacted in their mathematics classrooms that most students will have opportunities, 
or not, to achieve the expectations of the standards. Studies of the enacted curriculum should 
focus on a variety of elements, including: the sequence and connectedness of content; the 
mathematical tasks students encounter; the demands of solving and making sense of those tasks; 
the material, technological, and social resources students have to do so; and the assessment of 
student learning for formative and summative purposes. 

We view studying the enacted curriculum as a bridge for understanding the connection, or lack 
of connection, between the channels of influence described in the NRC framework and the extent 
and variability in intended and unintended student outcomes. Status studies of the enacted 
curriculum are recommended to answer a set of questions parallel to those for curriculum 
materials, recognizing that the curriculum materials being used are only part of what contributes 
to the enacted curriculum. It is particularly important to investigate the extent to which variation 
in alignment to the CCSSM across classrooms results in different learning opportunities for 
various student subgroups. Studies of the enacted curriculum should address the following 
questions, in each case exploring variations across student groups and contexts: 

1. To what extent are the standards for mathematical content at the K-8 grade level, or 
conceptual category in high school, evident in the enacted curriculum with the 
intended focus, rigor, and depth of the CCSSM? What variations in enactment, 
including appropriate differentiation, are evident across student groups and contexts 
of schooling? 

2. To what extent does the approach to and sequence of content in the enacted 
curriculum reflect the standards progressions of the CCSSM? 

3. How and to what extent are the standards for mathematical practice evident in the 
enacted curriculum, and integrated with the standards for mathematical content? 
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D. Studies to Improve the Standards 

In addition to understanding the influence of the current version of the CCSSM, the 
implementation of this set of standards provides an opportunity to generate knowledge that can 
inform revisions to the CCSSM, as well as development of future sets of standards. 
Investigations for this purpose would focus on the theoretical assumptions upon which the 
CCSSM themselves are based, and the ways in which revisions to the CCSSM may take into 
account emerging research. For example, the notion of developing deep student understanding 
of key ideas over time is central to the CCSSM. However, even though the standards 
progressions included in the CCSSM were intended to be based on evidence of learning along 
particular content trajectories, and validated to the extent that the evidence could provide such 
support, the authors have indicated that the research base was very thin in some areas. As a 
result, in many instances the standards progressions are hypothesized rather than empirically- 
supported learning progressions. 

Research efforts should capitalize on the fact that assessments, including the consortia 
assessments, are being designed to test student understanding along the trajectories of these 
standards progressions. Given that large numbers of students will take the same assessments, it 
will be possible to assess the validity of the hypothesized learning progressions, and to test the 
efficacy of alternative curricular and instructional approaches to support learning along these 
progressions. Alternative hypothetical learning progressions that researchers may generate 
should also be tested, which may entail developing and implementing alternative instructional 
materials as well as additional assessment items. Results from research of this kind can then be 
incorporated into revised standards progressions. Similarly, there may be areas where student 
performance falls particularly short of expectations and/or where teachers indicate something is 
just not working, even when the standards are implemented as intended. Such cases would 
indicate a need to rethink the standards progressions, including the grade placement of particular 
ideas. 

The CCSSM do not, as yet, provide progressions of understanding for the standards for 
mathematical practice that would clearly look different at various grade levels. Research that 
hypothesizes and investigates progressions for the standards for mathematical practice should 
inform revisions to the CCSSM that provide greater guidance than is currently available for 
addressing this critical feature of the standards. 

Another important area for research relates to the application of the standards for mathematical 
practice in instruction focused on the content standards, which may be more readily 
accomplished in some content domains than others. Research to illuminate how to accomplish 
this integration, and how it may be different or similar from one content domain to another, will 
support revisions to the CCSSM that could provide a more nuanced presentation of the standards 
for mathematical practice. 

Given that the CCSSM initiative is intended to improve college and career readiness, research to 
support revisions to the standards needs to go beyond the 12th grade. It is not only a matter of 
whether specific content is part of the K-12 progression, but whether achieving the expectations 
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of that progression actually ends in college and career readiness. Finally, whether focused on K- 
12 or post-secondary outcomes, investigations designed to provide empirical evidence for the 
CCSSM should sample a range of locales and diverse groups of students to ensure that 
conclusions are representative across contexts and subgroups. 


Methodological Considerations 

Implementation of the suggested research agenda has the potential to generate a wealth of 
knowledge about the influence of the Common Core State Standards for Mathematics. Whether 
or not that potential is realized depends not only on the quality of individual studies, but also on 
the extent to which the knowledge that is generated can be aggregated across studies to provide a 
comprehensive picture. The following recommendations are intended to help ensure systematic 
cumulation of knowledge in the field, both specifically in regard to the influence of the CCSSM 
and, we believe, more generally as well. 

• Theories of action should be used to provide structure for a set of studies. 

Specification of theories of action describing hypothesized pathways of influence of the CCSSM 
would help guide the design and interpretation of research (NCTM Research Committee, 2010; 
Weiss, 1997). Conducting research to investigate the various links along the various pathways of 
influence provides the means to amass chains of empirical evidence. In this way, sets of studies 
examining impacts and effects within different parts of the system can be combined to trace the 
nature and conditions of influence of the standards over time. Other studies could examine 
influence via a longitudinal, dynamic view, to follow hypothesized pathways of influence within 
a particular context conducted over longer periods of time. As the evidentiary base grows, the 
theory of action should be revisited both to: (1) synthesize knowledge (what has been learned 
about the relationships and pathways of influence the model posits and what still needs to be 
shored up through additional study), and (2) refine the model where evidence suggests it is 
under- specified or incorrect. 

• A data infrastructure needs to be created and managed. 

The widespread adoption of the CCSSM offers potential for research within and across many 
contexts, and on a large scale, drawing on comparable data. This potential can be realized only if 
a well-conceptualized, broadly accessible infrastructure is created for collecting, combining, and 
sharing data. Much more than a clearinghouse for data or a monitoring mechanism, the 
envisioned infrastructure would support decisions about directions and priorities for research. 
Tying both data and the results of studies to a specified theory of action as suggested above, the 
data infrastructure could serve to identify research needed to test promising approaches in new 
and diverse contexts; to examine why hypothesized relationships are or are not evident at various 
times and in various places; and to prompt studies that are needed to track changes over time 
once pre-requisite conditions or contingencies appear to be in place. Developing and managing a 
data infrastructure serving these functions would enable cumulation of research knowledge that 
can promote broader understanding of what policies, programs, and practices (in this case related 
to the implementations of the CCSSM) are and are not effective, for whom, and under what 
conditions. 
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• Common indicators of alignment need to be defined and used. 

In order to cumulate knowledge about the influence of standards, it is imperative that the field 
come to agreement about what distinguishes meaningful alignment with the CCSSM from 
superficial alignment and non-alignment. These indicators will be essential for assessing the 
extent of influence of the CCSSM on any part of the system. They will also be crucial for 
tracing influence between and among parts of the system, as conclusions about causal links and 
other dependencies need to be based on evidence that alignment with the standards in one part of 
the system is related to alignment in another. 

• Existing high quality instruments, and new instruments to be developed, need to be used 
more systematically in order for the field to understand the influence of the CCSSM. 

Research on the influence of the CCSSM requires instruments to measure the nature and extent 
of alignment of various components of the mathematics education system, for example, 
characterizing curriculum materials in terms of the aspects of the standards that are and are not 
evident. A variety of instruments for studying alignment with standards have been developed 
over the past 20 years. (See, for example, Council of Chief State School Officers (CCSSO) 
Surveys of Enacted Curriculum (SEC) Collaborative Project, 2005; Webb, 1997.) However, 
because these tools were developed before the CCSSM, they do not adequately address critical 
features such as the standards progressions or standards for mathematical practice. The 
revision/development and validation of the tools needed to pursue priority research questions on 
the CCSSM should itself be a priority for the field, perhaps using Race to the Top states and 
other early adopters as sites for this work. At the same time, the CCSSM may result in 
unintended consequences, e.g., decreased innovation in curriculum materials. These potential 
influences, too, must be accurately measured to be understood. Using sound measurement tools 
(existing or newly-developed), studies that employ the same instruments can be most easily 
compared and combined to examine similarities and differences in influence according to 
variations in interventions, contextual factors, target populations, and so on. At the same time, 
studies of the same relationship that employ different high quality instruments can provide a test 
of the robustness of findings, ensuring that they are not dependent on the idiosyncrasies of any 
particular measure. 

• Studies need to distinguish between alignment and quality. 

The CCSSM can be viewed as a set of hypotheses — if the system responds to the standards and 
mathematics education is provided as the standards expect, then improved student outcomes will 
result. At the same time, these standards do not encapsulate all that is known or hypothesized to 
be effective in improving the mathematics education system, so alignment with the standards 
cannot be equated with effectiveness. Rather, the influence of the CCSSM on teaching and 
learning clearly depends on factors that are not addressed in the standards. For example, two sets 
of curriculum materials, judged equally aligned with the standards for content and mathematical 
practice called for in the CCSSM, may have very different influences on classroom instruction 
and student learning because of other characteristics, such as educative features for teachers or 
incorporation of formative assessment techniques. In addition to attending to indicators of 
alignment, research on the influence of standards should consider other factors known to relate to 
effectiveness, as well as additional factors that may explain differential effectiveness. For 
informing future efforts, what turn out to be the strongest features of quality might be made 
explicit in plans for implementing standards. 
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• Planned studies need to be sensitive to the timeline of events. 

In planning research on the CCSSM, it is important to consider the timeframe and likelihood of 
influence. In jurisdictions that are moving quickly ahead with implementation of the CCSSM, 
including states that have received Race to the Top funding, it will be important to conduct 
research in the near term to investigate the relationship between the alignment of the enacted 
curriculum with the CCSSM and student outcomes. In contrast, it makes little sense to conduct a 
national observation study of the extent of classroom alignment with the CCSSM before aligned 
curriculum materials are readily available and efforts to implement the standards are well 
underway. Staying abreast of decisions that could alter the sequence of implementation phases is 
essential in executing relevant, timely, and informative research, especially because of the 
abundance of forces (federal officials, state officials, superintendents, teachers, the public, etc.) 
that can impact roll-out designs. 

• Phenomena need to be examined both “up close” and “at scale,” although not necessarily 
in the same studies or in that order. 

To build a robust knowledge base, research on the influence of the CCSSM should use a variety 
of approaches, including qualitative, quantitative, and mixed method studies. Research needs to 
include both “at scale” studies to document the extent of alignment of key components of the 
system with the CCSSM and “up close” studies to understand how that alignment/non-alignment 
came about, to explain differences in extent of alignment, and to investigate consequences. At- 
scale and up-close studies may be productively juxtaposed in a variety of ways. An at-scale 
study may uncover widely varying alignment of a key system component (e.g., adopted 
curriculum materials) across states, or across districts within a state. Up-close studies might 
follow to investigate the interpretations and factors that led to differences in alignment. Other 
up-close studies may examine important outcomes, such as teacher knowledge or classroom 
practice, to gain insight into the relationship between variations in alignment and possible effects 
on these downstream outcomes. Alternatively, an up-close study may reveal a particularly 
powerful influence on alignment in one or more contexts (e.g., the use of a specific, widely 
available tool for curriculum materials review) that can subsequently be investigated in at-scale 
studies to determine its use and importance more broadly. 

• Research needs to attend to context as well as content. 

Although it is not feasible to conduct research in every classroom in every community in the 
United States, it is vital that studies are conducted in a variety of contexts. In particular, given 
the vast intended reach of the CCSSM, research is needed to understand the influence of the 
standards in states where the CCSSM are fairly similar to and quite different from previous 
standards; in states with different policy contexts, in particular in relation to centralized versus 
local control; in different community types; in schools and classrooms with varying student 
demographics; and with teachers of different content backgrounds and years of experience. 
Similarly, although it is not necessary to conduct research on every content standard at every 
grade, it is important to consider multiple content areas in different grades to test similarities and 
differences in influence that may depend on nuances of the content itself, or its historical place in 
the U.S. curriculum. Additionally, studies are needed that address standards progressions across 
grades, to examine how this key feature of the CCSSM plays out at various levels of the system. 
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• Threats to validity need to be addressed through complementary studies. 

In social science, there is no such thing as a perfect study, one that avoids all threats to both 
internal and external validity. A stronger knowledge base will result from sets of studies that 
together address the major threats to validity for the priority research questions, so that the 
unaddressed threats in some studies are the addressed threats in others. (In particular, although 
people who are working to implement the CCSSM can and should study the effectiveness of 
their efforts, it is important that key relationships also be investigated by people who “have no 
dog in the fight.”) By acknowledging weaknesses and their possible implications in 
publications, authors can help ensure that other studies can directly address those weaknesses 
through alternative designs. It is the complementary nature of research approaches that yields 
the strongest empirical support for knowledge claims. 

• Research needs to consider issues of attribution. 

To some extent, research that compares and contrasts efforts to improve student mathematical 
knowledge, and thus college and career readiness, in states that have and have not adopted the 
CCSSM will be helpful in understanding the influence of these standards. At the same time, to 
the extent that most publishers revise curriculum materials, and professional development and 
other efforts focus on supporting implementation of the CCSSM, it will be difficult to find 
appropriate comparison groups. If the CCSSM affect resources and programs that are used in 
both adopting and non-adopting states, comparisons between these two groups of states will be 
limited in their usefulness for establishing attribution of influences on student learning 
experiences and outcomes. 

• Research that applies different theoretical and conceptual frames is key to cumulating 
knowledge. 

The knowledge generated within a field of research is strengthened when studies of the same 
phenomena, or related phenomena, can be combined and compared. Pursuing a common 
research agenda provides increased opportunity to cumulate knowledge. Fields of research are 
also strengthened by the development, application, and testing of different theoretical and 
conceptual frames that shape studies and provide a basis for interpreting results. The pursuit of 
this priority research agenda must remain receptive to contributions from researchers bringing 
various theoretical and conceptual stances to their empirical work. At the same time, researchers 
must consider all theoretical and conceptual stances open to consideration, critique, and 
empirical examination. 

• Research is an essential component of “engineering for effectiveness.” 

Given that education is enormously complex, with a huge number of variables that could make a 
difference in the implementation of the CCSSM in different contexts, the notion of engineering 
for effectiveness is helpful, where “education is treated as an organizational system that seeks, 
and is expected, to improve continuously” (Confrey & Maloney, 201 1). The idea of a cycle of 
discovery (National Science Foundation, 2010) is also helpful in thinking about research on the 
influence of the CCSSM. Results of studies conducted in best case situations, on samples of 
convenience, or on other non-representative samples, can help identify hypotheses that can 
subsequently be tested with representative samples. Alternatively, larger but less intensive 
studies conducted with representative samples can identify patterns of influence, supporting 
more in-depth investigations to seek possible explanations for those patterns — what works, for 
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whom, under what conditions. In either case, in a continuous improvement model, research on 
the influence of the CCSSM would identify actions, conditions, and resources that support 
implementation; interventions would be designed accordingly, and in turn, studied. 


Summary and Implications 

This report describes a priority research agenda for studying the influence of the CCSSM, 
developed with substantial input from mathematics education and policy researchers. It 
recommends research using a variety of approaches, including in-depth case studies to 
understand how key components of the system are changing in response to the CCSSM; status 
studies to determine the extent to which system components are becoming aligned with the 
CCSSM; and studies to explore the consequences, both positive and negative, of the CCSSM 
initiative on mathematics teaching and on a variety of student outcomes, K-12 and beyond. The 
priority research agenda also includes some preliminary ideas for generating knowledge to refine 
the standards over time, and includes a discussion of methodological considerations to help 
ensure the quality and rigor of individual studies, and the steady cumulation of knowledge. 

The priority research agenda, while not comprehensive, is nevertheless an ambitious one. 
Implementing it will take a long-term commitment on the part of one or more funders. It will 
also take the energy and talents of researchers with a variety of backgrounds and interests 
working in a complementary fashion, building on one another’s efforts. We believe that such a 
coordinated research enterprise is warranted in order to generate knowledge that can be used to 
improve this and future efforts to develop and implement standards. 
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Sample Indicators of Alignment 


Indicators of teacher preparation and professional development programs aligned with the 
CCSSM 

• The mathematics content included in the teacher preparation/PD program is aligned with 
the Standards for Mathematical Content indicated by the CCSSM at the course/grade 
levels of participating teachers. 

• The mathematics content included in the teacher preparation/PD program is addressed at 
and beyond the level of rigor expected of students at the course/grade levels of 
participating teachers. 

• The teacher preparation/PD program emphasizes how the mathematics content taught at 
one course/grade level relates to what students have already learned, what will come next 
in that course/grade, and what will follow in the future. 

• The Standards for Mathematical Practice in the CCSSM are modeled when engaging 
teachers with the Standards for Mathematical Content in the teacher preparation/PD 
program. 

• The teacher preparation/PD program includes explicit discussion of the Standards for 
Mathematical Practice indicated by the CCSSM at the course/grade levels of participating 
teachers and how teachers can promote their students’ development of these practices. 

• The teacher preparation/PD program explicitly addresses how to connect the Standards 
for Mathematical Practice to the Standards for Mathematical Content in instruction and in 
assessing student learning. 

Indicators of curriculum materials for students and guidance for teachers aligned with the 
CCSSM 

• K-12 curriculum materials for a course/grade address all of the Standards for 
Mathematical Content indicated by the CCSSM. 

• The mathematics content covered in the curriculum materials reflect the level of rigor 
indicated by the CCSSM. 

• The mathematics content included in the curriculum materials for a course/grade level 
builds on the Standards for Mathematical Content developed in previous course/grade 
levels. 

• K-12 curriculum materials provide guidance for teachers on how the mathematics 
content being considered relates to what students have already learned, what will come 
next in that course/grade, and what will follow in the future. 

• K-12 curriculum materials make explicit the Standards for Mathematical Practice 
indicated by the CCSSM. 

• K-12 curriculum materials incorporate the use of the Standards for Mathematical 
Practice indicated by the CCSSM when engaging students with the Standards for 
Mathematical Content, providing examples, activities, and tasks that highlight particular 
Standards for Mathematical Practice. 

• K-12 curriculum materials provide guidance for teachers about how to incorporate the 
Standards for Mathematical Practice while focusing on the Standards for Mathematical 
Content indicated by the CCSSM. 
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Indicators of K-12 student assessments and formative assessment tools/approaches aligned 
to the CCSSM 

• Student assessments measure attainment of the Standards for Mathematical Content 
indicated by the CCSSM for the designated course/grade. 

• Student assessments cover all of the Standards for Mathematical Content. 

• Student assessments for a particular course/grade reflect the level of rigor indicated by 
the CCSSM. 

• Student assessments are mapped to the progression of knowledge and skill development 
articulated within and across grades/courses and measure progress in student learning. 

• Formative assessment tools/approaches are linked to specific Standards for Mathematical 
Content in the CCSSM. 

• Formative assessment tools/approaches provide information about students’ development 
along a standards progression. 

• Student assessments include items that require students to apply the Standards for 
Mathematical Practice in the context of the Standards for Mathematical Content for the 
designated course/grade. 

• Formative assessment tools/approaches provide evidence of students’ understanding and 
application of the Standards for Mathematical Content and use of the Standards for 
Mathematical Practice referenced to where they stand in the standards progression. 
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Lines of Inquiry and Review Feedback 


Project staff used the notes from the various group discussions, and the participants’ feedback on 
the ideas that were presented, to develop a draft of a priority research agenda for studying the 
influence of the CCSSM on mathematics education in the United States. We identified 12 “lines 
of inquiry,” representing key leverage points, including “products” such as curriculum materials 
and professional development programs; “processes” such as state transition strategies; and 
“outcomes” such as college and career readiness. Figure 1 shows these lines of inquiry mapped 
to the NRC conceptual framework for research on the influence of national standards. 


Although in response to reviewers’ feedback we decided to both reorganize and streamline the 
priority research agenda, we thought it would be helpful to make available to the field not only 
the initial lines of inquiry, but also reviewers’ concerns about them. The lines of inquiry from 
the draft report are shown below, along with a sense of the reviewers’ feedback, typically shared 
in the form of composites of reviewers’ comments, and in some cases direct quotes. Note that 
we did not include comments like “Great point.” “Yes, yes, yes!” “Well said” and the like. 


We recognize that there is a fair amount of overlap among these lines of inquiry, and a particular component of the 
education system may be addressed in multiple lines of inquiry. For example, curriculum materials (the focus of the 
first proposed line of inquiry) might come into play in research on teacher preparation/professional development 
programs where teachers explore how particular curriculum materials can be used to address the CCSSM. 

Similarly, extent of alignment of curriculum materials with the CCSSM is likely to be a key factor in studies of 
classroom practice, of state transition strategies, etc. 
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Contextual 

Forces 

• Politicians and 

Policy Makers 

(5.6.7) 

• Public (7) 

• Business and 

Industry (7) 

• Professional 

Organisations (3. 6) 



Priority Lines of Inquiry 

1. Curriculum Materials 

2 Teacher Preparation Programs 

3 Teacher Professional Development Programs 

4 Consortia Assessments 

5 State. District, and School Leader Capacity 

6 System "Resources' for Implementation 

7 State Transition Strategies 

8 Teachers' Interpretations, Perspectives, and Plans tor Classroom Practice 
9. Classroom Practice 

10 K-12 Student Outcomes 

1 1 Post-secondary Education and Outcomes 
12. Future Modifications to the CCSSM 


Figure 1. A conceptual map showing the lines of inquiry mapped to the NRC Framework for Research on the Influence of National Standards Document 
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1. Curriculum Materials 

Curriculum materials are a major force in determining what gets taught in the K-12 classroom, 
as teachers often use their textbooks/programs as a guide (Weiss, Pasley, Smith, Banilower & 
Heck, 2003). As would be expected, developers of both commercially-generated textbooks and 
programs created in response to recommendations in the National Council of Teachers of 
Mathematics curriculum standards (National Council of Teachers of Mathematics 1989, 2000) 
have taken note of the huge market now created by common state standards. 

This line of inquiry includes understanding how curriculum materials are addressing the 
CCSSM."' Research in this area should focus on modifications to existing materials in light of 
the CCSSM, and the development of new curriculum materials. In particular, shifts in the nature 
and organization of content — grade-level placement, degree of emphasis, and treatment within 
standards progressions — compared to previous sets of standards should be examined. Also, the 
ways that the standards for mathematical practice are incorporated and integrated with content 
should be investigated. Because the CCSSM suggest two different pathways for the secondary 
grades, specific attention is needed regarding how the CCSSM are treated in curriculum 
materials designed for high school classrooms. 

In understanding the influence of the CCSSM on curriculum materials, it will also be important 
to analyze resources for teachers. Research should focus on how guidance for teachers in 
curriculum materials explicate content that has shifted due to the CCSSM, whether it is new 
topics to be taught at the grade level; different treatment of topics that have been typically taught 
at the grade level; or the expectations for explicit treatment or integration of the standards for 
mathematical practice. Similarly, research should examine how and to what extent materials 
intended for teachers explicate standards progressions, including how they underscore what 
students can be expected to have addressed in previous grades; how student understanding is 
intended to be developed in a particular grade; and what will come next. Another important 
target for research will be the nature of guidance for support or remediation when students’ pre- 


5 Given that there are relatively few major publishers of K-12 textbooks/programs, it would be feasible to examine 
all of the published mathematics textbooks/programs available for purchase at a particular point in time, especially if 
the research focus was on a subset of content areas; at the very least, there would be the possibility of including all 
published materials in a sampling frame. In contrast, we anticipate that there will be a very large number of 
supplemental materials marketed to teachers as aligned with the CCSSM. The important questions for research are 
whether teachers are able to select from the myriad of available materials those that are both high in quality and well 
aligned with the CCSSM, and to use them in a way that does not detract from the intended coherence of the 
curriculum. Accordingly, rather than including research on the alignment of supplemental materials with the 
CCSSM here, with some notion of sampling among them, we have included research on the alignment/use of 
supplemental materials in the lines of inquiry related to plans for, and enactment of classroom practice. 
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requisite knowledge and skills are not sufficient for addressing the grade level learning goals, or 
when their learning is not adequately progressing toward those goals. 

As curriculum materials that purport to be aligned with the CCSSM become available, and are 
being used in classrooms, research should seek to describe and explain relationships among the 
nature and extent of alignment of curriculum materials; their use in classrooms; and student 
outcomes. Equity questions should be paramount in such research, both because strongly aligned 
and potentially effective materials may be differentially available across contexts, or their 
effectiveness may differ by context or for various student subpopulations. 

Priority research questions in this line of inquiry: 

> How are existing curriculum materials being revised and new curriculum materials being 
developed to align with the CCSSM, including materials for traditional and integrated 
pathways at the high school level? What accounts for variation in the nature and extent of 
alignment? 

> In materials that purport to be aligned with the CCSSM, how much and what kinds of 
support are provided for teachers about how to address the key features of the CCSSM, 
e.g., standards progressions, and in particular how to assist students who are having 
difficulty with achieving the grade/course- specific standards? 

> What are the relationships among the extent of alignment with the CCSSM of curriculum 
materials, the enacted curriculum, and student outcomes? What explains/mediates these 
relationships in various contexts and for students from various subpopulations? 

> To what extent are aligned curriculum materials that are shown to be effective available 
to, and accessed by, districts and schools in various contexts serving students from 
various subpopulations? 

Reviewer Feedback 

You probably should define what you mean by curriculum materials. Do they include embedded 
assessments, end of chapter questions, formal tests, on-line and open source resources? 

“What about new delivery mechanisms — technology effects, wireless devices, cloud computing? 
Since these are likely to co-emerge, how should they be treated?” 

“Given that publishers are moving to digital forms of textbooks, it will be important to consider 
the types and formats of materials developed in response to CCSSM that may differ from 
currently available materials that are “paper bound”. That is, are authors/publishers using the 
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new technologies to in some way to enhance the alignment to CCSSM or make the linkages 
more direct/visible?” 

You haven’t mentioned the “magic 15%;” depending on what states add, how different it is from 
state to state, and how publishers respond, textbooks may not reflect the goal of fewer standards. 

“You may have made a decision to stay general here, but could also include some more specific 
ideas [for what should be studied] such as changes in mathematical representations or models.” 

How do curriculum authors view these standards, and what do they do when they believe that the 
CCSSM are “incoherent or wrong in topic placement/progression?” 

“Another high-priority research question concerns the consistency and collaboration across states 
with respect to which curriculum materials are being adopted.” 

It will be important to see the extent to which curriculum materials include formative assessment 
questions/tasks and provide guidance to teachers for “dealing with difficult issues with differing 
learners about focal topics to keep students on course with understanding - differentiating 
instruction within the CCSSM standards.” 

How are the CCSSM affecting decisions about whether to use a course- specific or integrated 
pathway at the high school level? 

It would be important to first determine the effectiveness of curriculum materials that are and are 
not aligned with the CCSSM — when implemented as intended, do they produce the desired 
learning, and can they be implemented as intended by typical teachers — and then determine if 
they are available to various subpopulations. 

There is a need to also look at available materials that do not align but are still used. “For 
instance, it will be interesting to see whether materials at the K-8 level will even mention 
calculators when the CCSSM do not.. . .1 think any analysis should allow for the fact that some 
classes and some schools may perform well even if their materials do not align with the CCSSM, 
and study of those classes and schools and the materials they use should be part of the larger 
study. In particular, one can imagine that top-performing schools and top-performing classes in 
many other schools will go well beyond the CCSSM.” 

2. Teacher Preparation Programs 

The quality of mathematics teaching and learning depends in large measure on having a well 
prepared teaching force. University pre-service programs typically focus on the content in their 
state standards. So, for example, recommendations by national groups for improving teacher 
preparation are interpreted in light of each particular state’s requirements. With common 
standards across states, one would expect that mathematics teacher preparation programs would 
become more similar as well. 
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The purpose of this line of inquiry is to determine how the CCSSM are, in fact, influencing 
teacher preparation programs, including the nature and extent of alignment with the CCSSM of 
both pre-service education programs and alternative teacher certification programs such as Teach 
for America. 

Research on the influence of the CCSSM on the teacher preparation programs themselves is 
recommended as an initial priority, including looking at a variety of programs to understand how 
and why alignment came about (or did not come about). Focused, in-depth inquiries into the 
process behind the incorporation of the CCSSM into specific teacher preparation programs may 
provide insight into the ways in which teacher preparation programs make curriculum decisions 
and modify existing structures to accommodate changes in standards. A thorough examination 
would address whether and how mathematics content that is covered in teacher preparation 
programs is aligned with the CCSSM, with particular attention to content that differs from 
previous standards, content that is receiving greater or lesser emphasis, and content that is 
organized differently. 

It will also be important to look at whether opportunities to experience aligned preparation 
programs are available for prospective teachers who will be teaching across a wide range of 
contexts, serving students from various demographic groups. Research efforts should include 
teacher preparation programs that vary by institutional home, grade range focus, program 
requirements, and the level at which they supply teachers to the highest needs districts. As new 
teachers complete preparation programs of different types in different settings, it will be 
important to examine outcomes among their graduates, specifically their knowledge and skills 
for teaching the CCSSM, as well as their beliefs about the importance and appropriateness of the 
expectations of the CCSSM. 

In later years, as the supply of teachers who were explicitly prepared for teaching the CCSSM 
increases, researchers should extend the range of outcomes to explore. For example, it will be 
important to determine the extent to which teachers are able to move more readily from one state 
to another, and whether district hiring policies favor teachers whose preparation was aligned with 
the CCSSM. 

Priority research questions in this line of inquiry: 

> To what extent and in what ways are pre-service and alternative teacher certification 
programs aligned with the CCSSM? What accounts for differences in extent of alignment? 

> What is the relationship between alignment of teacher preparation programs and new 
teachers’ knowledge, dispositions, and skills for teaching the CCSSM? What 
explains/mediates this relationship? 
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> What is the relationship between teacher preparation programs’ extent of alignment with 
the CCSSM and the likelihood that graduates will be hired in various contexts and be able 
to move more readily from state to state? 

Reviewer Feedback 

Need to look at whether state policy, including teacher preparation program 
approval/accreditation polices and teacher licensure tests become aligned with the CCSSM and 
help ensure that teachers have the opportunity to address all of the mathematics content they will 
be expected to teach. 

It would be interesting to study “how faculty members think differently about program design 
and, perhaps, look for more detailed cross-state collaborations than were possible when guided 
by the grade -band standards of NCTM.” 

Content courses for prospective mathematics teachers rarely focus on the content of K-12 
education. The idea that common standards across states will result in more similarity among 
teacher preparation programs doesn’t ring true; programs within individual states currently vary 
quite a bit, and they have the same standards. 

’’Teacher preparation programs are seldom designed to prepare teachers for work with the kind 
of detailed curriculum specifications that are laid out in the CCSSM. Thus the recommended 
research in this stream seems low priority in comparison with those related to curriculum 
content/sequence and assessment. The only exception might be to study how, if at all, teacher 
preparation programs focus on strategies for developing mathematical practice objectives at 
various levels of schooling.” 

You need to focus on mathematics classes and perhaps mathematics methods classes; asking 
about teacher preparation “programs” may be interpreted as focusing on how teachers are taught 
to teach. 

Research should include the extent to which higher education faculties revamp their classes to 
reflect CCSSM content and standards, and the processes used in achieving that alignment. Do 
the CCSSM foster better cooperation between mathematics education and mathematics faculty? 

Community colleges should be included in the research agenda, as prospective teachers often 
take content and even methods courses at those institutions. 

“It seems that some recognition should be given the challenge that prospective elementary 
teachers face in learning about detailed standards in Literacy, Mathematics, and soon Science.” 

How about the extent to which teacher education materials, e.g., methods books, content books, 
ancillary materials such as videos, are aligned with the CCSSM? And the extent to which the 
enactment, not just the course designs, reflect the CCSSM? 
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There needs to be a question about what pre-service teachers learn about the CCSSM in their 
preparation programs. 

Why stop with new teachers’ knowledge and skills? We need to link to their classroom practice 
and to their students’ outcomes as well. 

The question about district hiring of graduates of CCSSM-aligned preparation programs is “a 
real stretch,” hardly a priority for research. 

3. Teacher Professional Development Programs 

Even if teacher preparation programs became aligned with the CCSSM immediately, there are 
several million teachers of mathematics who are already in K-12 classrooms, many of whom 
will remain in the mathematics teaching force for quite some time. If the CCSSM are to be 
implemented at scale in the foreseeable future, the current mathematics teaching work force will 
need opportunities to develop the knowledge and skills to enact CCSSM-aligned curriculum 
materials. 

It will be important to target investigations on mathematics content topics that reflect various 
kinds of differences — new topics, different grade level placement, changes in approach, and 
more nuanced differences for familiar topics — in order to understand the influence of the 
CCSSM on professional development, as the changes that would be expected are different in 
these various scenarios. Two pivotal professional development-related junctures in the roll out of 
the CCSSM are the influence of CCSSM on professional development program design/ 
implementation, and the relationship between teacher participation in CCSSM-aligned 
professional development and teacher knowledge and practice. 

As with the teacher preparation line of inquiry, it is important to understand how and why 
various professional development programs become aligned with the CCSSM. In addition, 
it is important to note that there are many different players involved in designing and 
implementing mathematics professional development in the United States, including states, 
districts, colleges and universities, non-profit organizations, and a variety of commercial 
vendors. Accordingly, research to understand the influence of the CCSSM on professional 
development needs to explore the alignment of programs provided by a wide variety of players, 
identifying who is being affected across grade levels, teacher demographics, and community 
contexts. 

Professional development has the potential to influence teachers’ knowledge skills, and 
dispositions, but not all professional development programs realize that potential. As 
professional development programs are developed and revised to align with the CCSSM, 
researchers should assess what aspects of the professional development experience connect to 
changes in teachers’ knowledge, skills, or dispositions. 
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As more and more professional development programs that are intended to align with the 
CCSSM are implemented, it will be important to investigate the extent to which participating 
teachers are knowledgeable about the standards, and are willing and able to implement them in 
the classroom. Since a goal of the CCSSM is for all students to have access to coherent, rigorous 
mathematics, from an equity perspective, it is particularly important to look at teachers’ access to 
well-aligned professional development across contexts and student populations to determine the 
extent to which schools and districts with disparate levels of resources have teachers who 
become knowledgeable about the standards. Ultimately, research should trace the influence of 
the CCSSM from participation in professional development, to effects on teachers’ knowledge 
and skills, to implementation in the classroom, and finally to student outcomes. 

Priority research questions in this line of inquiry: 

> To what extent and in what ways are professional development courses, workshops, and 
school-based programs such as professional learning communities (PLCs) aligned with the 
CCSSM? What accounts for differences in extent of alignment? 

> What are the relationships among the extent of alignment of professional development with 
the CCSSM; teachers’ knowledge, skills, dispositions; the enacted curriculum; and student 
outcomes? What explains/mediates these relationships in various contexts and for teachers 
serving students from various subpopulations? 

> To what extent are aligned professional development programs that are shown to be 
effective available to, and accessed by, teachers of mathematics across a wide range of 
contexts, serving students from various subpopulations? 

Reviewer Feedback 

The CCSSM are going to put considerable pressure on the mathematics knowledge of teachers at 
the elementary and especially the middle grade levels; the lead-in does not do justice to the 
magnitude of the teacher capacity problem. 

“Do we know what knowledge and skills teachers need to develop? If not, how will we identify 
them?” 

Research question 3 needs criteria for effectiveness: Teacher knowledge? Quality of their 
teaching? Extent of student learning? 

It would be important to consider the role of teacher unions in the design and implementation of 
PD programs aligned with the CCSSM, and how state and district PD policy affects teacher 
participation in PD programs. 

Research needs to distinguish between PD design and implementation. First, we need to 
understand how the designers of PD are attempting to address the CCSSM, and the extent to 
which having common standards is leading to common designs. Then we need to study 


Horizon Research, Inc. 


C-9 


November 2011 



implementation to see if the facilitators model the mathematical practices in the PD; one could 
envision a PD session where someone lectured about the different mathematical practices. 

It will be important to consider the use of “affordances of technology for effective scale up of 
PD. . . . Does the scale of transition required by CCSS stimulate new thinking, new approaches to 
scalable PD?” 

To the extent that teachers need an in-depth understanding of mathematics in order to teach it 
well, it will be important to study whether mathematics PD programs are effective in deepening 
teacher content knowledge, identify the characteristics of effective mathematics PD programs, 
and determine whether teachers have access to them, whether or not those programs are closely 
aligned with the CCSSM. 

‘‘What is underdeveloped in the paragraphs above and in this cluster of research questions the 
need to structure the work of teaching so that professional learning happens naturally, 
purposefully, and even daily. This is the idea is behind professional learning communities, 
lesson study, and other initiatives. This movement complicates research about professional 
development, but I (and many others) believe that we are unlikely to make significant 
improvements in mathematics instruction as long as professional development is mostly viewed 
as an external force acting on a school. What is really needed, from a state and district 
perspective, is a strategic (rather than ad hoc) constellation of professional learning opportunities 
for and among teachers. So let’s have a research question that provokes study of the 
relationships among such opportunities.” 

“While it is almost certainly true that bringing teachers on board with the content and spirit of 
CCSSM is essential if the new standards are to be put into practice, the new standards are not 
really about teaching strategies. Of course we want to know what works in teacher PD. But 
we’ve always been concerned about that question. Since the field can’t study all of the questions 
that one might have about mathematics education, this area (like teacher preparation) seems a 
low priority for investments in research.” 

4. Consortia Assessments 

The request for proposals that resulted in two awards - to the Partnership for Assessment of 
Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium 
- called for assessments that would not only measure student learning gains and the effectiveness 
of teachers and schools, but could also be used to provide feedback for teachers to use in 
improving instruction. Especially if there are high stakes involved, these assessments have 
significant potential to drive changes, directly and indirectly, in teaching and learning. And it is 
clear that there are many concerns about the potential consequences of these assessments. 

Testing experts have cautioned against using a single assessment for any high stakes purpose, or 
for both formative and summative purposes. 

There is much to be learned about this grand experiment in common high stakes assessments that 
will help inform the design of future assessment systems in relation to standards, and thus it is 
important to document the development process and its consequences. The developers of the 
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consortia assessments have their hands full to get them ready for use in 2014-15 as planned; 
documenting the rationale for each of the myriad of decisions they are making will inevitably get 
lower priority than moving forward with the design and implementation process. People who are 
independent observers can be expected to be more objective in their documentation than those 
who are part of the decision-making process. For both of these reasons, to help ensure that the 
field can learn as much as possible, we recommend that researchers be embedded within the 
consortia assessment development groups as soon as possible to document design decisions and 
their rationale. 

During the development process, it will be important for research to examine how the unique 
features of the CCSSM, especially the standards progressions and the intended integration of 
standards for mathematical content and mathematical practice, influence the design of the 
formative and summative assessments. In addition, research should examine other factors 
affecting the design of the assessments including the affordances and challenges of computer- 
enabled testing, the proposed uses of the assessments for summative and formative purposes, and 
the demands of accountability and decision-making. 

The importance of the assessments to the implementation of the CCSSM at scale cannot be over- 
emphasized. Once the assessments are completed, it will be critical to describe the alignment of 
the mathematics content/mathematical practices addressed in those assessments with the CCSSM 
for each grade/course, as well as across grades. In addition, because validity of the assessment 
result reports will be affected by the design of the assessments, it will be important to study the 
extent to which the assessments are valid for the ways they are being used, both intended and 
unintended, e.g., for making consequential decisions for individual students, and for teacher and 
school evaluation. 

Priority research questions in this line of inquiry: 

> How do the consortia assessments measure student understanding of mathematics content 
and mathematical practices, and their ability to integrate the two? 

> How do the assessment systems track student attainment of the mathematics addressed in 
the standards progressions, both within and across grades? 

> How are accountability demands, e.g., for teacher and school evaluation, affecting the 
design and proposed uses of the consortia assessments? 

> How do the consortia assessments capitalize on affordances of computer-enabled 
administration? How are the consortia assessments addressing challenges associated with 
computer-enabled testing, both in design (e.g., computer adaptive tests) and administration 
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(e.g., student access to/experience with necessary technology across a wide range of 
contexts)? 

> To what extent are the consortia assessments valid for the ways they are being used? 
Reviewer Feedback 

It is important both to document the development process including the trade-offs made in 
designing the tests AND the consequences of the development process in the sense of the quality 
of the product produced. 

It will be important for, “the developers of performance standards/common core assessments to 
provide a rigorous definition of rigor.” 

More problematical is the fact that almost certainly, a significant part of the consortia 
assessments will be straight content - solve for x, simplify, etc. Surely some publisher will 
produce books that will in fact help students to be successful on these types of items, but will not 
help them learn mathematics concepts at a deeper level. That success may very well drive out 
more conceptual approaches that get at the heart of the practices; perhaps leading to de facto 
tracking, with such books being used for the low performing kids. 

“I think these progressions are overplayed, i.e., they are viewed here as having more effect than 
they will have. Every state standards document and every textbook series has a rather careful 
progression with respect to arithmetic at the elementary school level and algebra at the high 
school level, the areas in which the CCSSM have the most and longest progressions. In the other 
areas, the CCSSM do not have coherent learning progressions; the curriculum is quite uneven.” 

“I am not sure I know what it means to ‘integrate’ [understanding of mathematics content and 
mathematical practices.] That seems to imply that it might be possible to teach mathematical 
practices independent of learning mathematics content. Instead, I view them as the practices, or 
habits of mind, that need to be almost second nature to a mathematically proficient person. The 
CCSSM speaks of the practices as ways students “engage with” the mathematics content, i.e., it 
is about how one does mathematics, demonstrates that they are mathematically proficient (in the 
sense of Adding it Up) or engages in the NCTM process standards.” 

In what ways do the assessments address issues of equity, such as being available in students’ 
native language or allowing teachers to read the items to students whose reading level is below 
that needed to read the items? 

It seems like it would be good to add a priority research question something like this: “How are 
designs of the assessments driving change in curricula, instructional materials, teacher 
professional development efforts, and classroom practices?” 

What is the influence of the common assessments on decisions made by teachers regarding the 
sequence and intensity of instruction on various topics? 
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Do teachers, administrators, others feel that they should be teaching to the tests? Are outcomes 
on the tests seen as the most important thing or are they one component of a more nuanced view 
of education? Are there any changes in views as CCSSM is phased in? 

Add: How are the consortia assessments designed to serve accountability and/or formative and 
improvement purposes?. 

Add: How are the assessments actually used? What are their consequences? Based on this 
evaluation, how can the assessments be improved? 

There is much more to be discussed: 1) whether assessments become the proxy standards (as 
they are now); 2) many design issues in assessments (how the assessments are designed to 
provide feedback on mastery of standards; the breadth of content sampling; rigor of content; 
connections across domains and clusters and focus on structure; how scores are reported out; use 
of performance tasks, and professional development for teachers to score them; item release 
policies; etc.) 

Need to consider more than just the consortia assessments. Interim/benchmark assessments that 
will claim alignment with the CCSSM even before 2014; assessments that accompany 
curriculum materials 

I think the discussion of assessment needs to go beyond the Consortia Assessments. There are 
many states that may not sign up for the tests. One of the two consortia has recently backed off 
from building in benchmark assessments, in part because of concerns by states that the 
benchmarks will drive decisions that should be left up to the states/local districts. And states and 
local districts may well include other assessments. So I would have this section look at 
Assessment Systems that states and localities are developing, and their alignment with the 
CCSSM. 

The Consortia will struggle with the age-old problem of what content and skills can reasonably 
be assessed in a short period of time. So it will be important to look at what content and skills 
are included in (1) test specs; and then (2) test questions. These tests could lead to the same 
narrowing of the curriculum as current state tests. 


If we are looking at pass or don't graduate tests, especially at the high-school level, then we will 
almost certainly be back where we are with NCLB, i.e. lowering the passing rate or somehow 
exempting schools and districts. And this will happen in UNcommon ways, with different states 
reacting differently to political and social pressures. . ..I would love to track the tests to see 
whether they get easier or more rote memory driven or have lower passing grades over time. 

What about the question of whether proficiency on CCSM predicts college/career readiness? . 
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I don’t see how studying the validity of assessment use is a part of agenda for understanding the 
influence of standards; ditto for computer-enabled administration. These seem distant from 
studying influence. 

5. State, District, and School Leader Capacity 

High-quality implementation of the CCSSM at scale will require a great deal of capacity 
throughout the system, not only on the part of teachers of mathematics, but also among leaders at 
the state, district, and school levels. For example, quite a few states have processes for 
evaluating curriculum materials, and educators at the local level are asked to select textbooks 
from an approved list resulting from such evaluations. Given the central role of curriculum 
materials in the classroom, the extent to which content being addressed in a given state is 
consistent with the CCSSM may well depend on the ability of the people who develop and apply 
the curriculum materials evaluation criteria to distinguish between materials that are 
meaningfully versus superficially aligned with the standards. 

School administrators are charged with being instructional leaders in their buildings, and in that 
role might be expected to conduct classroom observations for teacher evaluation purposes, 
ensure articulation of the standards progression across grades, and monitor the use of evidence 
for improving teaching and learning. Consequently, it would be important for school 
administrators to be familiar with key features of the CCSSM and know what to look for that 
would indicate that the content and mathematical practices are aligned with the standards. In 
response to their perceptions of the needs of practitioners, policy makers allocate financial 
resources, such as funds for professional development, to facilitate the implementation process. 
Research in this area will help the field understand (a) what knowledge, skills, and processes are 
important at various levels of the system in order to have high-quality implementation of these 
particular standards, and (b) how those capacities can be developed most effectively given 
limited time and resources. 

Implementation of the CCSSM both depends on, and provides an opportunity to develop, the 
capacity of the mathematics education system, so it will be important to assess the capacity at 
each level of the education system in relation to the CCSSM over time. In addition, research 
should examine the extent to which these capacities are being developed across a range of 
contexts. 

Priority research questions in this line of inquiry: 

> What knowledge and skills are being targeted in efforts to increase the capacity of state, 
district, and school leaders to facilitate implementation of the CCSSM? 

> What strategies are being used to develop these capacities for leaders at various levels of 
the system? 
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> What is the relationship between the nature and extent of leaders’ capacities and their 
effectiveness in carrying out various roles in implementing the CCSSM? What 
explains/mediates these relationships in various contexts and for leaders serving various 
student subpopulations? 

> What strategies for developing capacity appear to be promising across a variety of 
contexts? 

Reviewer Feedback 

We need to know whether leaders are aware of the CCSSM and how well they 
understand them before we try to determine how capable they are of doing their part 
towards implementation. 

“It is not clear why you are using the term capacity here. Capacity to do what?” 

It would be helpful to be more specific in the research questions — leaders need to be able 
to coherently observe classrooms, interpret assessment data, support professional learning 
communities, talk to parents, share a vision. 

The narrative uses evaluation of curriculum materials as an example; that seems to be a 
“smaller issue” than the selection and delivery of professional development, including 
PLCs and other school-based learning opportunities. 

Your examples focus on supervision and enforcement rather than the softer side of 
leadership— creating conditions that enable learning for groups of teachers. Also, some of 
the leaders in schools aren't supervisors— or administrators. 

Teacher leaders are also worth studying. 

Implementation of the CCSSM will also depend on the awareness, knowledge, and skills of 
people outside the K-12 system, in particular higher education faculty who teach mathematics 
and mathematics education courses, provide professional development to in-service teachers, sit 
on textbook adoption committees, etc. 

Need to study not only what capacity is being developed, but by whom, e.g., who is providing 
PD for state superintendents, principals, etc., and how what is being targeted relates to what 
knowledge and skills are actually needed; focusing on identifying, placing, or growing leaders at 
strategic places in the system and attending to principals and other leaders who do not have 
mathematics expertise and the extent to which they draw mathematics expertise to the decision- 
making table. 

Need to explore the extent to which capacities are being developed through collaborations across 
schools, districts, and states now that the CCSSM provide common standards across those 
contexts. 
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The important question is not about “promising” strategies but about “effective” strategies. 

“There is little unique to CCSSM about the need to develop capacity, and to do so strategically; 
consequently this area should be a low priority research investment.” 

6. System “Resources” for Implementation 

The CCSSM were introduced into a system that includes a myriad of support structures for 
mathematics (and other) education. For example, the U.S. Department of Education has 
supported a national network of ten regional laboratories that, among other efforts, provided 
access to materials to address educational needs in their region. Colleges, universities, and non- 
profit organizations have developed numerous resources, often with funding from the National 
Science Foundation. Additionally, individual states may make available print and web-based 
resources to assist districts/schools/ teachers; intermediate service agencies within states may do 
the same for districts, variously targeting district- and school-level administrators, or individual 
teachers; districts may develop resources for schools and teachers; schools or departments within 
schools may develop resources for teachers; and professional associations may develop resources 
for their constituencies. Any or all of these support agents may respond to the release of the 
CCSSM by modifying existing programs/ resources and developing new ones to support 
implementation of this set of standards. 

The Mathematics Common Core Coalition (MC ) was formed specifically to facilitate successful 
implementation of the CCSSM. MC is a partnership of a number of key organizations-the 
National Council of Teachers of Mathematics, the National Council of Supervisors of 
Mathematics, the Association of Mathematics Teacher Educators, the Association of State 
Supervisors of Mathematics, the Council of Chief State School Officers, the Smarter Balanced 
Assessment Coalition, and the Partnership for the Assessment of Readiness for College and 
Career. The Coalition has set up a “go to” website with links to all of the CCSSM-related 
information and resources that these organizations are providing to their members and to the 
public. Given that the combined reach of these organizations is very large, it will be important to 
assess the quality and alignment of the likely widely-used resources posted on the site. 

The purpose of this line of inquiry is to examine key resources, those that are likely to exert 
considerable leverage-in terms of their alignment with the CCSSM as well as their quality, 
accessibility, and use. For example, the National Council of Supervisors of Mathematics and the 
Council of Chief State Schools Officers are collaborating in the development of tools for 
selecting instructional materials aligned with the CCSSM. If, as anticipated, these tools are 
broadly disseminated and widely used, they are likely to have a major influence on both the 
developers and consumers of instructional materials. First, it will be important to determine if 
these guidelines are able to distinguish between materials that are superficially and meaningfully 
aligned with the CCSSM. In addition, there are important considerations that go beyond 
alignment with the CCSSM, e.g., the extent to which instructional materials assist teachers in 
monitoring student understanding. Consequently, it will be important to determine the extent to 
which other research- supported elements of effective instructional materials are included in the 
instructional materials selection tool, and in any case, whether use of these tools leads to the 
selection of high quality materials. 
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Similarly, the CCSSM are likely to be considered in the revision of The Mathematical Education 
of Teachers (CBMS, 2001) currently underway, and that document is expected to have a major 
influence on the design of teacher preparation and professional development programs. It will be 
important to analyze the revised document in terms of both alignment with the CCSSM and 
consistency with current understanding of effective teacher preparation/professional 
development. 

In later years, as these resources are made available to the field, it will become critical to 
determine the extent to which their target audiences are aware of them, find them helpful, and are 
able to use them as intended, as well as whether they are accessible and useful across a variety of 
contexts. And as additional resources are developed and disseminated, similar research should be 
conducted on those that exert the most leverage on the system, either because they are being used 
directly in a large number of places, or because they are influencing the people, programs, or 
practices that in turn have a broad influence. Another key research strand examining longer-term 
impacts should look at the extent to which increased commonalities across states, created by 
uniform standards, are influencing the nature, quality, and quantity of the resources available to 
support implementation of the CCSSM. 

Studies within this line of inquiry should begin immediately to assess early uptake and use of 
system resources. Findings from this research questions should inform resource providers in 
terms of identifying necessary modifications to the resources and ensuring that resources are 
readily available to implementers across states and throughout the system. 

Priority research questions in this line of inquiry: 

> To what extent and in what ways are potentially high leverage system “resources” aligned 
with the CCSSM, e.g., tools for selecting instructional materials, and recommendations for 
teacher development? 

> To what extent are these resources consistent with prior research about effective 
implementation that is not explicitly addressed in CCSSM documents? 

> To what extent do common standards influence the nature, quality, and quantity of the 
resources available for implementation? 

> To what extent are the intended target audiences in various contexts aware of these 
resources, and perceive them as useful? 

> How extensively and well are these resources used, and by whom? 

Reviewer Feedback 

Parents are expected to play a key role in the extent of implementation as policy makers expect 
to see significant drops in test scores. It will be important to consider what resources are being 
provided, especially by groups like Ed Trust who have done this work in the past. 
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The research agenda needs to be sensitive to what turns out to be high leverage resources, not 
just the ones identified in advance as potentially high leverage. In particular, we need to 
determine what’s being learned from leveraging the affordances of technology. 

Add “and why” to the last research question. “What is it about the form, delivery of the resources 
and/or nature of the organizations providing them that makes them useable? For example, state- 
level NCTM chapters played an important role in the interpretation, dissemination and support of 
the NCTM standards in the 1990s.” 

These are all reasonable questions to ask, but not as critical as other parts of the research agenda. 

7. State Transition Strategies 

States that have adopted the CCSSM differ in their approaches to achieving implementation at 
scale. This line of inquiry focuses on the transition from one set of standards to another to 
inform both mid-course corrections in the implementation of the CCSSM and future efforts to 
implement new standards. 

Some states may focus initial efforts on mathematics content changes, holding off on addressing 
the mathematical practices until what they mean at different grades is more clearly defined. 
Others may start with the mathematical practices, anticipating that teachers will not be willing to 
move away from their state’s current content standards until consortia assessments are available 
and/or that integrating the mathematical practices will be easiest with familiar content. Other 
states may start with grades K-2, perhaps reasoning that teachers in non-tested grades will be 
more willing to begin teaching the new content right away. Another rationale for starting in the 
early elementary grades is that full implementation K-12 should be smoother if teachers in the 
higher grades can count on students having had opportunities in the lower grades to learn 
prerequisite content as specified in the CCSSM standards progressions. States that are moving 
quickly toward full implementation of the CCSSM in all grades (reportedly including Kentucky 
and New York) will provide the earliest opportunities for research on the effectiveness of 
particular transition strategies. 

Research on the factors that are influencing transition strategies will be important regardless of 
the time frame for full implementation of the CCSSM. Each state represents a unique set of 
constraints and opportunities, with districts that vary in the nature of the challenges they face, 
and the capacity and resources for improving mathematics education. We need a better 
understanding of what factors are taken into account; what parts of the system states plan to 
focus on, in what order, and in what depth; and how key stakeholders in the system, including 
private foundations such as Gates, and the business community and parent groups, are 
influencing transition plans. Over time, as these plans are implemented, there will be an 
opportunity to investigate the effectiveness of the various components of transition strategies 
within and across states. 

Priority research questions in this line of inquiry: 
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> To what extent, and how, do state transition strategies take into account the constraints and 
opportunities in their particular contexts? 

> How do key stakeholders (teachers, policy makers, private foundations, leaders in business 
and industry, parents, and the general public) exert influence on plans for CCSSM 
implementation? 

> How much emphasis is given to alignment with/preparation for teaching the CCSSM in 
selecting curriculum materials, hiring new teachers, designing professional development 
programs, etc.? Which aspects of alignment are given priority? 

> What transition strategies appear to be promising across a variety of state contexts? 
Reviewer Feedback 

Is there any evidence that the belief that it will be easier to integrate the mathematical practices 
with fa mi liar content? “Perhaps instruction on familiar content will be MOST resistant to 
change, just because it is so familiar (and sort of set in stone.)” 

The first question should point to the fact that there is diversity within states as well as across 
states. How do states take into account that their districts vary in important ways? 

Should the second question be bi-directional: How are key stakeholders influencing, and how 
are they influenced by, plans for CCSSM implementation? 

There needs to be a question about transition of state assessment programs to the consortium 
assessments. 

Race to the Top is a huge factor in state transitions, and needs to be considered in the research 
agenda. 

It will be important to study district transition strategies as well. 

As was suggested for case studies of the consortia assessment development, it will be important 
to embed researchers in case studies of state transitions, focusing on who makes decisions and 
how they are made. 

As part of understanding the transition process, it would be important to study the extent to 
which many states/districts are choosing an integrated approach to high school versus the course- 
by-course approach, and why. 

There is not enough attention to linking these transition strategies to the degree of 
implementation. 
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Include in this line of inquiry if/how districts, schools, and teachers are helping students catch up 
when they are behind as opposed to “relegating them to low track classes that serve mostly to 
slow them down even further.” 

“I think this section should be about state policies in support of CCSSM implementation in the 
different lines of inquiry. This would highlight where there are areas where states will differ 
(e.g., continuum of role in curriculum from state textbook adoption to being legislatively blocked 
from providing any state curriculum) and where state policy facilitates or inhibits CCSSM 
implementation. I have accordingly reordered and reworded the research questions below.” 

> How are states modifying policies to support implementation of the CCSSM? How do 
the states differ and why are they taking different paths? 

> What policy levers (e.g., mandates, incentives) are states using to influence which parts 
of the system (e.g., curriculum, teacher development, assessment) and at what level 
(state, local, classroom)? 

> How are key stakeholders (policy makers, leaders in business and industry, higher 
education, NGOs, parents, and the general public) exerting influence on plans for 
CCSSM implementation? 

> What strategies appear to be promising across a variety of state contexts? 

8. Teachers’ Interpretations, Perspectives, and Plans for Classroom Practice 

A number of lines of inquiry described thus far focus on tracing the influence of the CCSSM as 
they make their way to the classroom through the curriculum, teacher development, and 
assessment/accountability “channels.” But taking a systems perspective in considering how 
standards get to the classroom should not obscure the importance of individual teachers in 
determining what students experience in school mathematics. Teachers typically have 
considerable latitude in making class and homework assignments; setting expectations and 
conditions for students’ mathematical work; asking and answering questions during classroom 
discussions; grouping students for instruction; assessing student progress; and much more. And 
in fact, previous research has shown that classroom practice differs markedly even among classes 
within the same school that are using the same curriculum materials (Chval, Chavez, Reys, & 
Tarr, 2009; Kilpatrick, 2003). 

Individual teachers are at the heart of this line of inquiry, in particular how teachers are 
interpreting the CCSSM, how well prepared they feel to implement the CCSSM, their 
perspectives on the appropriateness of the CCSSM for their particular teaching situations, and 
how all of these factors impact their plans for classroom practice. 

In terms of interpretations, it will be important to determine the extent to which teachers 
understand the key elements of the CCSSM. For example, teachers may consider the content 
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standards more salient than the standards for mathematical practice, or vice versa, rather than 
considering the integration of the two as particularly important. Or teachers may focus on the 
mathematics content that is new for their grade range, perhaps overlooking more subtle shifts in 
the nature and focus of the content in topics they have previously taught at that grade. 

Equally important will be identifying the areas where teachers feel a need for additional 
assistance, whether in particular mathematics topics, or in understanding how to implement the 
standards for mathematical practice at their grade level. Information about what accounts for 
differences in teachers’ interpretations of the CCSSM and their perceptions of their preparedness 
for implementing these standards can be used to improve implementation of the CCSSM, for 
example in developing guidance documents for teachers and in planning professional 
development programs. In addition, it can help explain the extent to which plans for classroom 
practice are aligned with the CCSSM, including which textbooks/programs and supplemental 
materials they intend to use and how. 

Teachers’ views on whether the CCSSM are appropriate for their classes will likely influence 
their plans for instructions, and these views are likely to vary considerably by context. For 
example, teachers who have typically received ample school and district support for their 
mathematics programs may anticipate getting the support they need for implementing the 
CCSSM appropriately with their students. In contrast, in schools where teachers have not felt 
supported, especially in cases where students have performed poorly in mathematics, teachers 
may be concerned that standards billed as “rigorous” will be problematic. 

Teacher concerns may be particularly intense during the transition from their current state 
standards to the CCSSM, as students who have not had an opportunity to learn the mathematics 
called for in earlier grades may have difficulty mastering standards at the higher grades. Again, 
the results of research on teacher views of the appropriateness of the CCSSM for their teaching 
contexts can help explain the reluctance on the part of some teachers to implement these 
standards, and can also be used in considering how to provide additional support to teachers. 
Priority research questions in this line of inquiry: 

> To what extent do teachers understand the key elements of the CCSSM? What accounts for 
differences in teachers’ understanding? 

> What are teachers’ perceptions of their preparedness to implement the CCSSM in relation 
to particular mathematics topics, and the standards for mathematical practice in their 
courses/grade level? What accounts for differences in teachers’ perceived preparedness? 

> To what extent do teachers from a wide range of contexts serving students from various 
subpopulations consider the CCSSM to be appropriate for the students in their classes? 
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> What are the relationships among teachers’ understanding; perceived preparedness; beliefs 
about the appropriateness of the CCSSM for their students; and plans for classroom 
practice? What explains/mediates these relationships in a variety of contexts? 

Reviewer Feedback 

In the first research question, we want to know if teachers understand how the CCSSM differs 
from the current standards they are following. We probably also want to know whether teachers 
are willing to invest in learning about the CCSSM, or if they are waiting because “this too shall 
pass.” 

In terms of what accounts for differences in teachers’ understanding, it will be important to 
consider individual factors such as depth of teacher content knowledge, as well as collective 
factors related to school climate and policy. 

“Independent of whether teachers know what is recommended by the CCSSM, teacher 
mathematical knowledge is very important. But this seems missing from this line of inquiry. 
Shouldn’t that be THE most important question, i.e., what is an appropriate measure of teachers’ 
mathematical knowledge for teaching and how does that impact their ability to teach the 
mathematics recommended by the CSSSM?” 

The second research question doesn’t really address the key issue of integrating content and 
practices; perhaps ask what accounts for differences in teachers’ perceived preparedness to 
integrate the two. And in accounting for those differences, consider teacher self-efficacy as well 
as content knowledge and conceptions of the CCSSM and their rigor. 

This line of inquiry needs to address the fact that some teachers believe that the standards cannot 
be for all students. “This issue is especially problematic in districts that have been performing 
adequately for some time, getting most students (-80%) over the current proficiency bars. Too 
many teachers are willing to write off the other 20% of students — and the situation is especially 
troubling in high schools. But because high schools have long functioned as sorting 
mechanisms, we need to recognize that the idea of college and career readiness for all is a very 
big shift.” 

A key issue is “whether math teachers at all levels see a reason to interact more across levels, 
and perhaps support each other in mixed groups, such as learning communities of various types. 
Rather than improvements being driven "top down," could there be opportunities to develop 
greater autonomy within the mathematics teaching community? Or do things go the opposite 
direction and become even more top down, even more driven narrowly by supervision and 
assessments?” 

Add: How are teachers interacting with cross-grade colleagues to ensure smooth transitions and 
adequate preparation for the next level? 
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Add: To what extent, and how, does having the same learning goals result in teachers at 
different sites sharing plans for, and lessons learned about, improving classroom practice? What 
conditions support teachers working collaboratively? 

9. The Enacted Curriculum* 

In order to influence student outcomes, the CCSSM need to first influence the enacted 
curriculum, changing the mathematics addressed, and the opportunities students have to learn 
that content. Early research efforts should focus on the nature and extent of alignment of the 
enacted curriculum to the CCSSM, and how and why that alignment came about. How the 
salient features of the CCSSM are manifest in the enacted curriculum is likely to vary widely. 

For example, in some classrooms the standards for mathematics content may be adhered to quite 
carefully, with teachers following school/district pacing guides organized along the lines of the 
CCSSM; in other classrooms the alignment may be less extensive. In some classrooms, only the 
teachers’ actions may reflect the mathematical practices, for instance the use of appropriate 
mathematical tools, while in others, students may be actively engaged in doing mathematics in 
ways that utilize the mathematical practices. With respect to standards progressions, some 
teachers may make explicit connections between the mathematics content that is being addressed 
in a particular lesson and what students experienced earlier that year or in previous years, while 
others may not. 

All of these types of variation can be anticipated to occur not just from one classroom to another, 
but across schools, districts, and states. Studies documenting systematic variations should 
investigate possible explanatory and mediating policy or program factors at these broader levels. 
At the same time, even within a school or for any particular teacher, the enacted curriculum in 
some topics areas may be more aligned with the CCSSM than in others, suggesting a need for 
research to study what factors, e.g., teacher knowledge, or use of supplemental materials, 
account for these differences. 

Large-scale studies to document the extent of alignment of the enacted curriculum (and the 
quality of instruction) within states or nationally are appropriate and important when curriculum 
materials aligned with the CCSSM are widely available and teachers have had an opportunity for 
initial preparation and/or professional development related to the standards. It will be important 
to disaggregate the results for different kinds of contexts, as well as by student, classroom, 
school, and district demographics. Understanding how the nature and extent of alignment of the 
enacted curriculum varies across contexts and for different student groups can then inform 
studies to explain the roots of those differences. 

Priority research questions in this line of inquiry: 

> To what extent and in what ways are the CCSSM being implemented in classrooms, 
specifically in relation to materials used, mathematics content addressed, use of 


In the draft report, this line of inquiry was labeled “Classroom Practice.” Reviewers noted that this designation 
was problematic, as the CCSSM are silent on teaching strategies. Here and elsewhere in the document, we now 
refer to this area as research on the enacted curriculum, focusing on what mathematics content and mathematical 
practices are addressed, and how much attention is paid to standards progressions. 
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mathematical practices in addressing content standards, and attention to standards 
progressions? What accounts for variations in implementation? 

> How does the nature and extent of alignment (and quality) of the enacted curriculum vary 
across contexts, and for various student subpopulations? What accounts for these 
variations? 

Reviewer Feedback 

We know there is a difference between “covering” a topic and providing students opportunity to 
learn that content. How will this line of inquiry handle that distinction? 

The first research question should include something about how the enacted curriculum has 
changed. 

“There doesn’t seem to be enough attention to high quality instructional materials scaffolding 
classroom practice, especially when the teacher edition provides substantial guidance. “What 
we’ve seen is that teachers in [one district] — not necessarily exemplary teachers — have a much 
higher ‘floor’ in their classroom activities than the teachers in the [other district], because of the 
affordances of the curricula they use. Even if you’re not that skilled, using one curriculum sets 
up you and your students to be engaging in richer activities (student-to-student discussions, for 
example, which ask for explanations) than another. So, how do you look for the affordances for 
classroom practice, and see how they play out?” 

What about teachers’ use of assessments, the extent to which they are aligned with the CCSSM. 

We need to look at how assessment info facilitated by CCSSM is being used by teachers and 
PLCs to improve mathematics instruction. 

“This reads pretty thinly relative to what we are learning about classroom practice — the role of 
stages of implementation (launch, explore, reflect) and the types of discourse and grouping and 
the use of formative assessment and possibly just in time instruction and real time data. I wonder 
if you want to again give a better sense of what we have learned about examining practice ... to 
remind the reader of the expectation of the context of such work on standards and how they 
interact with these types of investigations of practice.” 

“I want MUCH more detail here. 

First, we need classroom observation tools that are more attuned to the things called for 
in CCSSM (especially the practices) than current tools. 

Second, we need studies of change — close-up fine grained studies of how teachers 
evolve, and with what support. Think of David Cohen’s “Mrs. Oublier” study, revealing 
the distance between a teacher’s beliefs about her own practice (pure reform) and the 
reality (old practices still dominated) 

Third, we need theory and research on teachers’ developmental trajectories. So we can 
learn what’s realistically possible to support by way of teacher growth, as documented by 
honest-to-goodness changes in practice.” 
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10. K-12 Student Outcomes 

The ultimate goal of the CCSSM is to improve student outcomes at scale, narrowing historic 
achievement gaps and ensuring that all students leave high school well prepared. Student 
outcomes may be defined with the whole child in mind, including mathematical literacy goals, 
but also taking into account student efficacy, identity, or effort. If pedagogical changes result in 
different styles of interactions among students, or between students and teachers, there may well 
be an influence on outcomes such as sense of belonging, behavioral issues, or rates of 
absenteeism. 

Although studies can be mounted early on to see if aligned practice leads to the desired student 
outcomes - which might be considered existence proofs of the efficacy of the CCSSM - there is 
a good possibility that the “early adopters” will not be representative of the mathematics teaching 
force as a whole. Just as it seems appropriate to hold off on major studies of changes in 
classroom practice until the CCSSM have had time to permeate other aspects of the system, 
major studies of the relationship between classroom practice and student outcomes should be 
delayed until the CCSSM have had an opportunity to permeate the system. The consortia 
assessments that are being designed to align with the CCSSM are scheduled to be available in 
2014-15, and they will be an important — but certainly not the only — source of information about 
the relationship between CCSSM-influenced classroom practice and student achievement. Other 
outcome measures that have provided important information in the past can also contribute a 
great deal to our understanding of the influence of the CCSSM on students in the United States, 
including national studies (e.g., National Assessments of Educational Progress, Advanced 
Placement, International Baccalaureate, SAT, ACT), international studies, (e.g., Programme for 
International Student Assessment and Trends in International Mathematics and Science Study), 
and less extensive but more intensive measures (e.g.. Balanced Assessment in Mathematics). 

Research in this line of inquiry should focus on the relationship between alignment of classroom 
practice with the CCSSM and various student outcomes. In addition, research should examine 
the factors that might explain these findings. As noted earlier, the CCSSM focused on the 
mathematics that should be addressed, not how that mathematics should be taught. The CCSSM 
do not address, for example, what we know from prior research about means of communicating 
high expectations to students; the importance of ongoing monitoring of student understanding; 
etc. Accordingly, it is important to note, the enacted curriculum may be well aligned with the 
CCSSM but the instruction may be of poor quality, or the enacted curriculum may be poorly 
aligned with the CCSSM but the instruction may be high quality. Consequently, in 
understanding the influence of the CCSSM on student outcomes, it will be important to consider 
not only alignment of the enacted curriculum but also the extent to which classroom practice 
incorporates features of quality instruction. 
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Given the CCSSM’s goals for all students to have access to coherent, rigorous mathematics 
content, differences in outcomes related to equity concerns need to be a focus of studies of the 
relationship between the enacted curriculum and student outcomes. For example, it will be 
vitally important to distinguish between persistent differences in achievement due to different 
instructional experiences, versus differences in achievement due to differential effectiveness of 
the same instructional experiences for various groups of students. Finally, in the long-term, it 
will be important to study the extent to which the ultimate goal of the CCSSM has been attained 
by examining whether student outcomes on a variety of measures are improving at scale and 
historic achievement gaps narrowed. 

Priority research questions in this line of inquiry: 

> What are the relationships between the extent of alignment of the enacted curriculum with 
the CCSSM (in terms of mathematics content, mathematical practices, emphasis on 
standards progressions) and particular student outcomes? What explains/mediates these 
relationships? 

> How do the relationships between alignment of the enacted curriculum with the CCSSM 
and student outcomes vary for different subpopulations? What explains/mediates these 
relationships? 

> Over time, to what extent is implementation of the CCSSM in various content areas/grade 
levels leading to improved student outcomes at scale, including narrowing historic 
achievement gaps? 

Reviewer Feedback 

You need to discuss the possibility of a whole host of outcome measures, cognitive, achievement 
orientation, student interest in mathematics, identity and agency, absenteeism, drop out rates, and 
career expectations. 

Add: What are models of CCSSM implementation that are effective in achieving various student 
outcomes? What do they look like and why do they work? 

Need to study outcomes in relation to tracking practices. 

Given that scores on a high-stakes test tend to improve over time (particularly in the early years 
of administration) as teachers become more and more familiar with the kinds of items that will 
appear and instruct their students accordingly. So a major additional question to ask is whether 
the changes in student performance from year to year are reflected on other tests, both those that 
are revised to be aligned with the CCSSM and those that are not tied to the CCSSM. 
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11. Post-secondary Education Preparedness and Outcomes 

A major impetus for developing and promulgating the CCSSM was concern expressed from 
various elements of K-12 education, higher education, government, and business and industry 
that the United States has fallen behind other nations in preparing its graduates for success 
beyond high school. Given this foundational emphasis on college- and career-readiness, a 
priority for research will be investigating whether variations in students’ K-12 experiences in 
terms of alignment with the CCSSM are related to differences in post-secondary preparedness. In 
fact, one of the influences of the CCSSM on mathematics education may be more attention to 
defining and measuring college and career readiness. 

Post-secondary educational intentions, enrollments, and completion; and career entry and 
advancement, especially in STEM fields, should be examined over time to determine the extent 
to which the CCSSM are supporting progress toward the goal of college and career readiness. 

Of particular interest will be differences in post- secondary outcomes for students who complete 
an integrated mathematics program in high school versus a traditional sequence, two alternative 
approaches described in the standards. And for equity purposes, it is vital that among these 
investigations are studies that consider variability in the outcomes among different subgroups of 
students. 

Although the CCSSM initiative does not extend to post- secondary education through any 
specific recommendations, its proponents anticipate that the experiences and readiness of high 
school graduates will affect higher education. For instance, if the CCSSM result in stronger 
learning on a more widespread basis, then institutions of higher education-from research 
universities and liberal arts colleges, to community colleges and technical schools-should have 
less need for remedial courses in mathematics, and there should be greater enrollment and 
success in credit-bearing courses. 

Whatever mechanisms might be needed to accomplish it, a critical long-term objective of the 
CCSSM is improving career readiness, both overall and in STEM-related employment. If the 
CCSSM are having the intended impact, employers across the spectrum will have access to 
better prepared employees, and there will be a larger and more diverse pool of people prepared 
for careers in STEM disciplines. 

Priority research questions in this line of inquiry: 

> To what extent are students becoming better prepared for post- secondary education (e.g., as 
evidenced by decreased enrollment in remedial courses) in a variety of types of institutions 
of higher education? How does student preparedness relate to their participation in CCSSM- 
aligned programs in their K-12 education, including both traditional and integrated high 
school mathematics programs? 
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> To what extent are institutions of higher education using evidence of attainment of the 
CCSSM for placement decisions in credit-bearing courses? How successful are the students 
who enroll in these courses, and how does this vary by type of institution and among student 
subpopulations? 

> To what extent are the CCSSM improving college and career readiness at scale, including 
for subpopulations that have historically been underrepresented in STEM post- secondary 
education and careers? 

Reviewer Feedback 

“I might add a question about whether mathematics instruction in colleges changes either due to 
CCSSM directly or to the changes in what knowledge students bring with them to college. In 
particular, does mathematics instruction for preservice teachers change? (As an aside, do we 
know whether mathematics instruction in colleges changed in response to greater technology 
expertise of students entering college?)” 

It is not at all clear that greater enrollment in college mathematics should be a consequence of 
improved mathematics programs at the K-12 level. College students often take only that which 
they are required to take. If, for example, most business majors at my university came to campus 
ready for our business calculus class (rather than college algebra), it would result in a reduction 
of enrollment in credit bearing courses. 

Expecting that there will be a more diverse pool of people prepared for careers in STEM 
disciplines implies an equity principle that I do not believe is part of the CCSSM. 

“I can’t resist noting that the CCSSM goal of preparing college-ready students takes as 
unproblematic the current college definition of readiness. Since the channeling of students into 
remedial college work is usually based on scores attained in placement tests whose validity is 
widely challenged, we need to keep an open mind about the definition of college ready.” 

“With respect to colleges, my understanding is that the consortia have made arrangements with a 
large number of colleges to accept a passing grade on a high-stakes high school test as a 'get out 
of remediation' card. . . .1 would love to see how kids who pass these tests actually do on 
standardized college placement exams. Of course, some percentage will fail, but how will that 
percentage compare with how students did in the pre-CCSSM days?” 

To what extent, then DO CCSSM skills relate to success in college? This is a test of the whole 
model. If it doesn’t get researched, then the whole effort is for naught. Now Career-readiness 
will be the real tough one. What subset of the CCSSM is needed for different (non-degree- 
required) careers? 

“There’s much more emphasis in these research questions on post- secondary education, almost 
to the exclusion of examining the impact on career readiness for those students who directly 
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enter the workforce upon completing high school. I recommend including at least one priority 
research question relating just to career readiness.” 

Add: How are the reports generated by the assessments of value and used? 

Add: In what ways is the CCSSM serving as a catalyst for conversations across high school and 
higher education in improving articulation, remediation rates, and mathematics instruction in 
both high school and college? 

Add: To what extent are businesses using evidence of attainment on the consortia assessments 
for hiring decisions? 

12. Future Modifications to the CCSSM 

In addition to understanding the influence of the current version of the CCSSM, the 
implementation of this set of standards provides an opportunity to generate knowledge that can 
inform revisions to the CCSSM. This line of inquiry focuses on the investigations of theoretical 
assumptions upon which the CCSSM themselves are based and the ways in which revisions to 
the CCSSM may take into account contemporary research. For example, the notion of 
developing student understanding of key ideas over time is central to the CCSSM. However, 
even though the standards progressions included in the CCSSM were intended to be based on 
evidence of learning along particular content trajectories, and validated to the extent that the 
evidence could provide such support, the authors have indicated that the research base was very 
thin in some areas. As a result, in many instances the standards progressions are hypothesized 
rather than empirically-supported learning progressions. 

Research efforts should capitalize on the fact that the consortia assessments are being designed to 
test student understanding along the trajectories of these standards progressions. Given that large 
numbers of students will take the same assessments, it will be possible to assess the validity of 
the hypothesized learning progressions, and to test the efficacy of alternative curricular and 
instructional approaches to support learning along these progressions. Alternative hypotheses 
that researchers may generate as learning progressions should also be tested, even if it means 
developing and administering additional items. Results from research of this kind can then be 
incorporated into revised standards progressions. Similarly, there may be areas where student 
performance falls particularly short of expectations even when the standards are implemented as 
intended, suggesting a need to rethink the grade placement of particular ideas. Investigations 
designed to provide empirical evidence for the CCSSM should sample diverse groups of students 
to ensure conclusions are representative across subpopulations. 

The CCSSM do not, as yet, provide progressions of understanding for the standards for 
mathematical practice. The standards for mathematical practice are designed to apply across the 
K-12 grades, but clearly would look different at various grade levels. And students are expected 
to develop greater fluency with these practices throughout their K-12 education. Research that 
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hypothesizes and investigates progressions for the standards for mathematical practice should 
inform revisions to the CCSSM that provide greater guidance than is currently available for 
addressing this critical feature of the standards. 

Another important area for research relates to the integration of the content standards with the 
standards for mathematical practice, which may be more readily accomplished in some content 
domains than others. Research to illuminate how to accomplish this integration, and how it may 
be different or similar from one content domain to another, will support revisions to the CCSSM 
that could provide a more nuanced presentation of the standards for mathematical practice. 

Priority research questions in this line of inquiry: 

> To what extent are the standards progressions in the CCSSM empirically supported as 
learning progressions leading to strong conceptual understanding and flexible application 
of knowledge? 

> What learning progressions can be hypothesized for students’ learning of the standards for 
mathematical practice? How are these progressions similar or different for various content 
areas? To what extent are these progressions empirically supported? 

> To the extent that standards progressions are not empirically supported: 

o Which standards should be re-assigned to different grade-level placements based 
on evidence of developmental appropriateness? 
o What gaps should be filled with additional standards that support students’ 
learning along each standards progression? 
o Which standards progressions should be replaced with alternative progressions or 
re-sequenced to support stronger learning, and how? 

> Considering students from various subpopulations and contexts, what learning 
opportunities support students’ learning of the CCSSM? 

Reviewer feedback 

It is important to recognize both that the consortia assessments are unlikely to provide the fine- 
grained information that will be needed and that it is not possible to test alternative learning 
progressions simply by giving a few additional test items in the consortia assessments. “As 
almost all proponents of learning progressions or learning trajectories admit, the progress of 
student learning is intimately bound up in the instructional experiences those students have. 

Thus to truly test alternative progressions that might suggest significant revisions of the first 
CCSSM, we need to permit and encourage research that develops and implements genuine 
alternative curricular and instructional trajectories.” 

“Some would argue that multiple progressions are possible (the fractions folks, for example, 
don’t agree on which progression is the ‘right’ one), so it seems that there could be a question 
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about alternative progressions (not as replacements for existing progressions but as alternatives 
to them).” 

Given the large number of possible reasonable sequences for learning mathematics, these 
research questions are misleading, seeming to assume that there is some magic order towards 
which we should be striving. 

‘‘What would constitute evidence that a standards progression is not empirically supported? The 
CCSSM do not specify in any detail the level of achievement that would be considered 
satisfactory.” 

Teachers indicating that something is just not working, and test scores that show something is 
wrong should be considered evidence of “developmental inappropriateness.” 

What about the trajectory to college readiness? It’s not only a matter of whether specific content 
is part of the trajectory, but whether that trajectory actually ends in college readiness. 

Given that the CCSSM initiative is intended to improve college and career readiness, research to 
support revisions to the standards needs to go beyond the 12 th grade. 

Research should also focus on “the care and feeding of the standards as things move forward.” 
Among the questions to be addressed: Is this governance structure viable? Who should weigh 
evidence, and when, in making decisions about how to improve the CCSSM? How should the 
process of developing standards be modified for future standards? 
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Other Research Areas of Interest for 
Understanding the Influence of the CCSSM 


Given their widespread adoption, the CCSSM are likely to influence virtually every component of the 
mathematics education system in the United States. The priority research agenda suggests studies in a 
subset of areas that were considered particularly important. If resources are available, investigations in 
other key areas would provide a fuller understanding of the influence of the CCSSM, knowledge that 
would be useful in learning about implementation of these standards at scale, and the design and 
implementation of future standards. 

Teacher Preparation 

> To what extent and in what ways are mathematics/mathematics education classes in pre- 
service and alternative teacher certification programs aligned with the content and 
mathematical practices specified in the CCSSM? What accounts for differences in extent 
of alignment? 

> What is the relationship between alignment of teacher preparation programs and: (a) new 
teachers’ knowledge, dispositions, and skills for teaching the CCSSM; (b) their 
mathematics teaching; and (c) student outcomes? What explains/mediates these 
relationships? 

Teacher Professional Development 

> To what extent and in what ways are professional development materials, courses, 
workshops, and school-based programs such as professional learning communities 
aligned with the CCSSM? What accounts for differences in extent of alignment? 

> How effective are professional development programs in enhancing teachers’ knowledge 
and skills; improving their mathematics teaching; and improving student outcomes? What 
explains/mediates these relationships in various contexts and for teachers serving students 
from various subpopulations? 

Institutions of Higher Education 

> To what extent are the CCSSM serving as a catalyst for conversations about articulation between 
high school and higher education? 

> To what extent arc mathematics faculty members at institutions of higher education, including 
community colleges, aware of the CCSSM? What implications, if any, do they see for their 
mathematics courses? 

> How arc the design and implementation of college mathematics courses changing in response to 
the CCSSM and, over time, to the changes in mathematics backgrounds of entering students? 

Professional Societies 

> How are professional societies that are involved in mathematics education responding to the 
CCSSM in terms of services and resources provided to their members? 

> To what extent are these services/resources being used, by whom, and with what results? 

Supplemental Materials 

> What supplemental materials, including assessment tools and on-line resources, arc being 
marketed as aligned with the CCSSM, and in what ways are they aligned? 

> How are these materials being used, by whom, and with what results? 
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